2012 Data Discussion: Difference between revisions

From canSAS
Line 53: Line 53:
** Qy must be the Q component along the Y axis
** Qy must be the Q component along the Y axis
** Qz must be the Q component along the Z axis
** Qz must be the Q component along the Z axis
** It is not clear if GISAXS geometries will work with this Q definition.  Requires expert input from the community.  It is thought that two SASdata groups will be used to describe the two Q mappings on the same intensity data.  (Needs to be confirmed.)
* Thus, we can drop the @Q attribute as redundant
* Thus, we can drop the @Q attribute as redundant
* transformation to/from other coordinate descriptions (such as from Peter Boesecke) is needed
** IUCr CIF format has a generic mapping algorithm (ask Herbert Bernstein)
* We must allow for multiple SASdata groups in a SASentry
* We must allow for multiple SASdata groups in a SASentry
** This is easy for XML and makes for writing easy validation procedures.
** This is easy for XML and makes for writing easy validation procedures.

Revision as of 12:57, 30 July 2012

Minutes

Saturday Afternoon Working Group Session

Sunday Morning Discussion Session

  • How many implementations?
    • Need at least one reference implementation by end of meeting
    • Shouldn't preclude multiple possible implementations. RG noted that if they follow the same data structure then they should be "trivially" interchangeable.
  • SESANS? Spin-Echo? XPCS?
    • Out of scope for our remit.
  • Next step -> science examples.
  • How to incorporate complimentary data e.g. uv spectra
  • Space for calibration information
  • Space for process data


Sunday Afternoon Working Group Session

  • Based on morning discussion...
    • Redefined a proposed minimum spec outline:
      • 2012-minimum.png
    • Redefined a proposed minimum recommended spec outline:
      • 2012-recommended-minimum.png
  • Generated a proposal for usage and format.

See: 2012_Data_Discussion_Examples

Monday Afternoon Working Group Session

  • expect other standards group to supply dictionary of terms
    • start with dictionary from cansas1d/1.l0 standard
  • Definition of Q is based on a cartesian coordinate with the sample at the center as defined in the cansas1d/1.0 standard
    • All Q objects when present must have the same shape (rank and dimensions)
    • At least one of these four data objects (Q, Qx, Qy, Qz) must be present
    • Q, when present, must be |Q|
    • Qx must be the Q component along the X axis
    • Qy must be the Q component along the Y axis
    • Qz must be the Q component along the Z axis
    • It is not clear if GISAXS geometries will work with this Q definition. Requires expert input from the community. It is thought that two SASdata groups will be used to describe the two Q mappings on the same intensity data. (Needs to be confirmed.)
  • Thus, we can drop the @Q attribute as redundant
  • transformation to/from other coordinate descriptions (such as from Peter Boesecke) is needed
    • IUCr CIF format has a generic mapping algorithm (ask Herbert Bernstein)
  • We must allow for multiple SASdata groups in a SASentry
    • This is easy for XML and makes for writing easy validation procedures.
    • HDF5 can't allow "SASdata" to be used as the object's name if more than one might be present. Needs a solution (NeXus chose one way to deal with this).

Agenda with Discussion

  • The following is the agenda of work posted under business for canSAS-2012. Please add comments in the appropriate section below:

1D Format

Agree a proposed extension of the current 1D standard

  • This is an extension proposed by Steve King (ISIS) that is required to enable, for example, t-o-f instruments to store auxillary wavelength-dependent non-I(Q) data in the same output files

Agree resultant changes to the "official" schema/stylesheet

SMK -  
For clarity, this extension was proposed after adoption of the existing version of the standard. Ratification of this extension at CanSAS-2012 would give facilities/software developers the necessary confidence to implement it.
SMK -  
This proposed extension would allow more-or-less any non-I(Q) data to be included in a CanSAS-1D data file under a suitable foreign namespace. That is remarkable flexibility, and it does not require any significant revision of the existing standard. However, it could be argued that certain types of non-I(Q) data are so integral to a SAS experiment that they should instead be given explicit SASXML tags within the standard? Transmission data is a good example of this: the existing standard explicitly includes a tag for the transmission value from a monochromatic measurement (SASsample\transmission) but makes no provision for transmission values from white beam measurements. The proposed foreign namespace extension will make such provision possible but only as an extension, not as a formal part of the standard. Is that acceptable? The downside of explicitly adding new SASXML tags to the existing standard is that it would require issuing a new version of the standard.
ARJN -  
I would recommend that during this meeting that the current format then be 'parked', the new 2D format should be flexible enough to handle 1D and 2D data.
SMK -  
Agreed.
ARR -  
Perhaps we should be cautious about suggesting no further development for the existing 1-D format. Unless there is scope for adapting to future needs, it will be seen as not suitable for new uses. This, of course, does not imply that there have to be active plans for further changes at present.
AJJ -  
As to this particular case, I suggest that since SMK and I have it done, that we accept the additional tags, call it v1.1 and move on. If other extensions arise later and become "widely" used, then we can incorporate them as they arise. This is the key benefit of using an extensible format.
ARJN -  
Just as a matter of interest, why is it necessary to include transmission measurements for TOF instruments? Does the analysis depend on these values? Not saying it's not necessary, just wondering about why it is useful.

2D Format

Define minimum information necessary for reduced data

  • definition of what reduced data is and hence the problems we should address.
  • review previous discussions on 2D
  • considerations specific to 2D reduced data
  • consider forward looking issues such as
    • grazing incidence
    • event mode analysis

Suggest format framework

  • NeXus extension, canSAS 1D extension, other, with brief discussion of the reason for the choice (including options considered, pros and cons of each, and final weighing)
ARJN -  
Having looked at both those Nexus definitions it is not clear to me how:
    • multiple detectors are handled.
    • What happens if the detector pixels are not grid shaped?
    • how does one stuff multiple datasets (e.g. from event mode acquisition)?

Remember, all we need is instrument independent information in this file, nothing else.

Andyfaff 12:45, 27 July 2012 (CDT)
  • James Hester (an expert in the CIF format) has compelling ideas on the subject of common data files. CIF format notes Hester
  • Ron Ghosh has laid out details of a proposal to use HDF5/Nexus : HDF5 Notes Ghosh
  • Peter Boesecke misses definitions of parameters that are needed to describe small-angle and wide-angle scattering experiments. Parameters like "Distance", "Center", "Rotation" have only a meaning when they are part of a geometrical model, otherwise they are useless. He has collected parameters that are required for SAS/WAS experiments with position sensitive detectors (1D and 2D). When saving experimental data it should be checked that these parameters are either implicitly or explicitly described by the supplied metadata: SX_parametrization.

Guide to how to make implementation easy.

Create straw format suitable for test/demonstration use and convert some test data for such a demonstration.

Provide a plan for presentation at SAS 2012

Retrieved from "https://wiki.cansas.org/index.php?title=2012_Data_Discussion&oldid=1206"