Operators

The data operator classes are meant to organize the various mathematic manipulations that are performed on the data containers. For example, many of the operations performed in the existing HOPS3 code-base (such as the application of a priori phase calibration) are relatively trivial linear transformations applied to the visibility data. However, in the HOPS3 code base these simple operations have become intertwined with a large amount of control logic which obscures the basic data pathway (e.g see source/c_src/fourfit_libs/ffsearch/src/norm_fx.c)

Most unary or binary operations that are to applied to visibility or other data residing in a MHO_TableContainers object such as scaling, multiplication, transposition, summation, Fourier transformation, etc. have in HOPS4 been reorganized into individual classes inheriting from the same operator interface. A uniform class interface allows these data operators to be composed or modified to create more complicated composite operators or strung together and called in an ordered fashion in order to accomplish data pipelines of arbitrary complexity. An additional advantage of encapsulating individual operations is that (coupled with the data container extensions) most SIMD parallel-processing extensions (e.g. CUDA/OpenCL plugins) used to accelerate data processing can be made opaque to the caller.

The following code gives a brief sketch of the class templates generalizing the data operators.

class MHO_Operator
{
    public:
        MHO_Operator();
        virtual ~MHO_Operator();
        virtual bool Initialize() = 0;
        virtual bool Execute() = 0;

        //name and priority (priority determines execution order within a category)
        void SetName(const std::string& name);
        std::string GetName() const;
        void SetPriority(double priority);
        double Priority() const;
};

template<class XArgType>
class MHO_UnaryOperator: public MHO_Operator
{
    public:
        MHO_UnaryOperator();
        virtual ~MHO_UnaryOperator();
        virtual void SetArgs(XArgType* in); //in-place
        virtual void SetArgs(const XArgType* in, XArgType* out); //out-of-place
};

template<class XArgType1, class XArgType2 = XArgType1, class XArgType3 = XArgType2>
class MHO_BinaryOperator: public MHO_Operator
{
   public:
       MHO_BinaryOperator();
       virtual ~MHO_BinaryOperator();
       virtual void SetArgs(const XArgType1* in1, const XArgType2* in2, XArgType3* out) //out-of-place
};

The common inheritance from the base class hops::MHO_Operator allows operators of various types to all be stored in an common container ( e.g. std::vector<MHO_Operator*>) so that once they are constructed they may be retrieved and intialized/executed in the appropriate order. The following code shows a brief code sketch demonstrating how two simple operators would be constructed, assigned arguments, initialized, and then executed in order.

//prexisting pointers to a particular data container
//data_type* array1;
//data_type* array2;

//storage for an ordered list of operators
std::vector< MHO_Operator* > operators;

//construct and configure two operations, and insert in list
//(MHO_ScalarMultiply is illustrative here; a similar effect can be achieved
//via MHO_FunctorBroadcaster with a scalar-multiply functor)
MHO_ScalarMultiply mult;
mult.SetFactor(2);
mult.SetArgs(array1);

MHO_SubSample sub;
sub.SetDimensionAndStride(0, 4);
sub.SetArgs(array1, array2);

operators.push_back(&mult);
operators.push_back(&sub);

//chained initialization of the operators
for(auto it = operators.begin(); it != operators.end(); it++)
{
    it->Initialize();
}

{ //some possible outer loop here...

    //chained execution of the operators
    for(auto it = operators.begin(); it != operators.end(); it++)
    {
        it->Execute();
    }

}
//array2 now contains the contents of array1, multiplied by 2
//but sampled only every 4th element in the 0-th dimension

The vast majority of the data operators are either unary or binary. Unary operators accepting either single input data container (to operate in-place), or both an input and output data container (of the same type) to operate out-of-place. Binary operators require two input data containers, and a single output data container (all not necessarily of the same type) to carry out an out-of-place execution. There are two additional categories of operator beyond unary/binary. These are ‘transforming’ and ‘inspecting’ operators. Transforming operators are specific form of unary operator which accept an input and output data container, that are not of the same type, and change the input into the output. While, inspecting operators do not modifying the input data container at all, but are used to compute some information from it (e.g. finding a maximum in an array). A fourth base class, MHO_UnaryInPlaceOperator, exists for unary operators that only support in-place operation (no out-of-place variant). While the aforementioned cases cover all of the use cases in HOPS4, the operator interface is very generic and for future use cases any number of input/output arguments is possible, as long as the underlying implementation provides the appropriate overloads.

Specific data operations

Below is an incomplete list of various data operations. A full specification of each operation is detailed in the subsequent pages.

MHO_ComplexConjugator

Class	MHO_ComplexConjugator
Operator Type	Functor (used via MHO_FunctorBroadcaster)
Template Parameters	A N dimensional array type
Argument Data Type	N dimensional array with complex double/float value type

Iterates over all values in an N dimensional array and applies the operation std::conj() to each element. This is a functor class (derived from MHO_UnaryFunctor, not from MHO_Operator) and must be wrapped by MHO_FunctorBroadcaster to be used as a pipeline operator; it supports both in-place and out-of-place operation via that wrapper.

MHO_CyclicRotator

Class	MHO_CyclicRotator
Operator Type	Unary
Template Parameters	The specific N dimensional array type
Configuration Parameters	Integer index of axis to rotate and integer offset for rotation size
Argument Data Type	N dimensional array with arbitrary trivially copyable type

Performs cyclic rotation upon the requested axis for the specified offset. A positive value of the rotation offset results in a right shift cyclic rotation, while a negative value results in a left shift cyclic rotation. This is a template class supporting both in-place and out-of-place operations.

MHO_FastFourierTransform

Class	MHO_FastFourierTransform
Operator Type	Unary
Template Parameters	XFloatType (floating point type)
Configuration Parameters	Direction of transform (forward/backward, follows FFTW convention)
Argument Data Type	One dimensional array with complex double/float value type

Performs a Fourier transform (or inverse transform) on the input array using an FFT algorithm. If the array size is a power of two, then either a Cooley-Tukey or Gentleman-Sande radix-2 algorithm will be applied. For all other sizes, the Bluestein/Chirp-Z algorithm is used. This is a template class supporting both in-place and out-of-place operations.

MHO_FunctorBroadcaster

Class	MHO_FunctorBroadcaster
Operator Type	Unary
Template Parameters	XArrayType (array type), XFunctorType (functor type)
Configuration Parameters	The unary functor class to be applied to each element
Argument Data Type	N dimensional array with any value type (must be acceptable to functor)

For every element in the array the functor operation will be applied. In the case of an out-of-place operation a copy will take place. This is a template class supporting both in-place and out-of-place operations.

MHO_MultidimensionalFastFourierTransform

Class	MHO_MultidimensionalFastFourierTransform
Operator Type	Unary
Template Parameters	XArgType (N dimensional array type)
Configuration Parameters	Indices of dimensions to transform (default all), direction (forward/backward)
Argument Data Type	N dimensional array with complex double/float value type

Executes a Fourier transform on the selected dimensions of the array using the native FFT calculator. The direction follows the convention of FFTW. This is a template class supporting both in-place and out-of-place operations.

MHO_MultidimensionalFastFourierTransformFFTW

Class	MHO_MultidimensionalFastFourierTransformFFTW
Operator Type	Unary
Template Parameters	XArgType (N dimensional array type)
Configuration Parameters	Indices of dimensions to transform (default all), direction (forward/backward)
Argument Data Type	N dimensional array with complex double/float value type

Executes a Fourier transform on the selected dimensions of the array using the FFTW library. The precise algorithm selected is determined by FFTW. The direction follows the convention of FFTW. This is a template class supporting both in-place and out-of-place operations.

MHO_Reducer

Class	MHO_Reducer
Operator Type	Unary
Template Parameters	XArrayType (array type), XFunctorType (reduction operation)
Configuration Parameters	Indices of dimensions to reduce, operation type (addition/multiplication)
Argument Data Type	N dimensional array with numerical value type

The input array will be reduced along the selected axes, and depending on the operation (addition or multiplication), the contents will be resized and replaced by the sum or product of the elements along that axis. Template class supporting both in-place (requires copy and resize) and out-of-place operations.

MHO_SubSample

Class	MHO_SubSample
Operator Type	Unary
Template Parameters	XArrayType (array type)
Configuration Parameters	Dimension index for sub-sampling, stride for element selection
Argument Data Type	N dimensional array with any value type

For a stride value of k, and dimension index j, the output array will be resized and populated in such a way that only every k-th element (along the j-th dimension) from the original array will remain. This is a template class supporting both in-place and out-of-place operations.

MHO_AbsoluteValue

Class	MHO_AbsoluteValue
Operator Type	Functor (used via MHO_FunctorBroadcaster)
Template Parameters	XArrayType (array type)
Configuration Parameters	None
Argument Data Type	N dimensional array with numerical value type

Applies the absolute value operation to every element of the input array. This is a functor class (derived from MHO_UnaryFunctor, not from MHO_Operator) and must be wrapped by MHO_FunctorBroadcaster to be used as a pipeline operator; it supports both in-place and out-of-place operation via that wrapper.

MHO_ElementTypeCaster

Class	MHO_ElementTypeCaster
Operator Type	Transforming
Template Parameters	XArrayType (array type), XNewType (target type)
Configuration Parameters	None
Argument Data Type	N dimensional array with any value type

Casts the elements of the input array from one type to another. Template class supporting both in-place and out-of-place operations.

MHO_EndZeroPadder

Class	MHO_EndZeroPadder
Operator Type	Unary
Template Parameters	XArrayType (array type)
Configuration Parameters	Dimension index and padding size
Argument Data Type	N dimensional array with numerical value type

Pads the end of the specified dimension with zeros. This is a template class supporting both in-place and out-of-place operations.

MHO_ExtremaSearch

Class	MHO_ExtremaSearch
Operator Type	Inspecting
Template Parameters	XArrayType (array type)
Configuration Parameters	Search type (minimum/maximum)
Argument Data Type	N dimensional array with numerical value type

Searches for the minimum or maximum value in the input array and returns the value and its location. This is a template class for inspecting array contents.

MHO_NaNMasker

Class	MHO_NaNMasker
Operator Type	Functor (used via MHO_FunctorBroadcaster)
Template Parameters	XArrayType (array type)
Configuration Parameters	Mask value to replace NaN with
Argument Data Type	N dimensional array with floating point value type

Replaces NaN values in the input array with a specified mask value. This is a functor class (derived from MHO_UnaryFunctor, not from MHO_Operator) and must be wrapped by MHO_FunctorBroadcaster to be used as a pipeline operator; it supports both in-place and out-of-place operation via that wrapper.

MHO_SelectRepack

Class	MHO_SelectRepack
Operator Type	Unary
Template Parameters	XArrayType (array type)
Configuration Parameters	Selection criteria and packing parameters
Argument Data Type	N dimensional array with any value type

Selects elements from the input array based on specified criteria and repacks them into a new array structure. This is a template class supporting data reorganization.

Compound data operations

On their own each of the specific data operations listed in the previous section are of limited utility. However, they can be composed to produce more useful manipulations of the data (e.g. MHO_NormFX). The advantage of composing complex operations via a series of simple operators is that more fine grained testing can be done at each sub-step to ensure it is operating correctly without involving the much more complicated process (as opposed to the all-or-nothing approach of the (HOPS3) function norm_fx.c).