lib_vdif module

Module with basic functions to read and write VDIF headers and data.

– VDIF Specification version [VD01]–
VLBI Data Interchange Format (VDIF) Specification
Release 1.0 (w/24 August 2009 corrections/clarifications highlighted)
Ratified 26 June 2009

Notes

-bitarray library still used for VDIF file generation.
Updated 2015.10.15:
-frame length (//8 when writing header, *8 when reading header)
-changed endiannes to be correct
2015.12.03: changed bits2int(bitsin)
2015.12.03: read_samples is now more efficient, but may not work with numbers of bits per sample that do not divide 32.
TO DO: implement in “read_samples” the functionality to read bits per sample that do not divide 32.
TO DO: testing on epoch and seconds conversions.
2016.09.21: bitarray reader implementation removed in favor of raw which is faster.
lib_vdif.bits2int(bitsin)[source]
lib_vdif.compute_range(word_size, bits_per_sample)[source]

Get starting positions in word for groups of bits_per_sample bits.


Example:

word_size=32
bits_per_sample=4
list(compute_range(word_size,bits_per_sample))
>>> [0, 4, 8, 12, 16, 20, 24, 28]
lib_vdif.create_header_vdif(seconds_fr=17, invalid=False, legacy=True, ref_epoch=15, frame_num=1, vdif_version=1, log_2_channels=1, frame_length=20000, data_type=0, bits_per_sample=2, thread_id=1, station_id=1)[source]

TO DO: check epoch...

lib_vdif.date_to_vdif(year, month, day, hour, minute, second, offset_seconds=0)[source]

Compute VDIF fields for epoch and seconds from date.

lib_vdif.disp_list(a)[source]
lib_vdif.get_vdif_num_frames(filename, packet_limit=-1, forced_packet_size=0, offset_bytes=0, only_offset_once=1, v=1)[source]

Get information from vdif file by processing headers. As many headers as packet_limit are processed. Each of the output parameters are vectors with all the possible values found.

See get_vdif_stats().
num_frames : int
number of frames read.

TO DO:

This is a modified version of get_vdif_stats(), merge with it.
lib_vdif.get_vdif_stats(filename, packet_limit=-1, forced_packet_size=0, offset_bytes=0, only_offset_once=1, v=1, short_output=0)[source]

Get information from vdif file by processing headers. As many headers as packet_limit are processed. Each of the output parameters are vectors with all the possible values found.

filename : str
path to VDIF file.
packet_limit : int
[default -1] maximum number of packets, if -1 no limit.
forced_packet_size : int
[default 0] if >0 will read this number of bytes instead those in the VDIF header.
offset_bytes : int
[default 0] skip this number of bytes before reading the first VDIF header.
only_offset_once : int
[default 1] if ==0 will skip offset_bytes before every frame to be read.
v : int
[default 1] verbose mode if 1.
short_output : int
[default 0] group very long vector (frames) if 1.
v_stations : list
list with unique ids found for stations in all frames read.
v_seconds : list
list with unique seconds found for all frames read.
v_frames : list
list with unique frame numbers for all frames read.
v_sizes : list
list with unique frame sizes for all frames read.

total_size : int total size for all frames read.

lib_vdif.int2bits(value)[source]
lib_vdif.print_header_mark5(sync_word, years, user_data, bit_t, frame_num, time_code_w1, time_code_w2, crcc)[source]

[DELETE]

Untested. Consider deleting.

lib_vdif.print_header_vdif(seconds_fr=17, invalid=False, legacy=True, ref_epoch=15, frame_num=1, vdif_version=1, log_2_channels=1, frame_length=20000, data_type=0, bits_per_sample=2, thread_id=1, station_id=1)[source]

Show header of vdif frame, for logging.

See read_header_vdif_from_raw() output.
lib_vdif.print_header_vdif_info(brief)[source]

Show stats results header.

lib_vdif.print_header_vdif_row(id=0, seconds_fr=17, invalid=False, legacy=True, ref_epoch=15, frame_num=1, vdif_version=1, log_2_channels=1, frame_length=20000, data_type=0, bits_per_sample=2, thread_id=1, station_id=1)[source]
lib_vdif.read_bytes_offset_file(f, n_bytes, v=0)[source]

Used to skip some offset when reading a binary file.

f : file handler
(sys.stdin).
n_words : int
number of words of type TYPE_WORD.
v : int
[0 by default] verbose mode if 1.
lib_vdif.read_header_mark5(header)[source]

[DELETE]

Untested. Consider deleting.

lib_vdif.read_header_vdif_from_raw(header)[source]

Reader vdif header.

header : numpy array with four np.unit32 words.
[seconds_fr,invalid,legacy,ref_epoch,frame_num,vdif_version,log_2_channels,

frame_length,data_type,bits_per_sample,thread_id,station_id]: where:

seconds_fr: VDIF frame seconds (integer).
invalid: VDIF invalid bit.
legacy: VDIF legacy bit.
ref_epoch: VDIF frame epoch (float with MJD (TBC)).
frame_num: VDIF frame number (integer).
vdif_version: VDIF frame version (integer).
log_2_channels: VDIF logarithm in base 2 of the number of channels in the frame.
frame_length: VDIF frame length (integer with number of bytes).
data_type: VDIF frame data type bit.
bits_per_sample: number of bits per sample.
thread_id: VDIF thread identifier field (integer).
station_id: VDIF station id field (integer).
lib_vdif.read_samples_from_raw(words, bits_per_sample, word_size=32)[source]

Extract samples from unit32 words into numpy array of integers.

words : numpy array of np.unit32. bits_per_sample : int word_size : int

Limitations:
bits per sample should be a divisor of word_size.
lib_vdif.read_vdif_frame(f, show_errors=0, forced_frame_length=0, offset_bytes=0, v=0, return_raw_header=0)[source]

Main routine for reading one VDIF frame.

f : file handler
input file handler (typically sys.stdin).
show_errors : int
[0 by default] display information on errors if 1.
forced_frame_length : int
[0 by default] number of bytes to read including header, if 0 will take value from header (recommended).
offset_bytes : int
[0 by default] number of bytes to skip before reading header.
v : int
[0 by default] verbosed mode if 1.
return_raw_header : int
[0 by default] if 0 returns a list of integers with the header fields, if 1 returns bytes.
header : list of int
list of integers with the fields in the VDIF header:
[seconds_fr,invalid,legacy,ref_epoch,frame_num,vdif_version,log_2_channels, | frame_length,data_type,bits_per_sample,thread_id,station_id].
See read_header_vdif_from_raw() for details.
If errors, header is None.
samples : 1D numpy array of int
sample components. E.g. for VDIF complex frame, [I0, Q0, I1, Q1, ...]
if errors, samples is None.
check_size_samples : int
1 if read as many samples as expected from the frame length (and rest of metadata), 0 otherwise.

Configuration:

HEADER_VDIF_WORDS
HEADER_BYTES
WORD_SIZE_BYTES

Procedure:

1. Read VDIF header.
2. Read rest of the frame based on frame length information in header.
lib_vdif.read_words_from_file_to_raw(f, n_words, v=0)[source]

Read binary file directly into numpy array.

f : file handler
(sys.stdin).
n_words : int
number of words of type TYPE_WORD.
v : int
[0 by default] verbose mode if 1.
words_array : numpy array with number->n_words type->TYPE_WORD words.

Configuration:

TYPE_WORD: [np.uint32 by default] type of words to read.
lib_vdif.reshape_samples(allsamples, data_type, samples_in_frame, num_channels)[source]

Corner-turning.

allsamples : 1D array
with all samples read from VDIF file.
data_type : int
0 for real, 1 for complex.
samples_in_frame : int
number of samples components per channel.
num_channels : int
number of channels.
samples : 2D array with num_channels rows, with each row containing only its associated samples.

Procedure:

# example
aa=list(range(0,20))
bb=np.reshape(aa,(-1,5,2))
cc=np.transpose(bb,(1,0,2))
dd=np.reshape(cc,(5,-1))
dd
lib_vdif.show_headers_vdif(filename, limit_frames=-1, skip_frames=-1, brief=0)[source]

Get tabulated information from vdif file headers. The first “limit_frames” after “skip_frames” are read.

filename : str
path to file.
limit_frames : int
maximum number of lines to display (-1 for no limit).
skip_frames : int
number of frames to skip before starting displaying frames (default <=0).
brief : int
1 to avoid displaying descriptions for columns.
N/A
lib_vdif.strbitarray2int(a)[source]
lib_vdif.vdif_epoch_seconds_to_epoch_seconds_datetime(epoch_six, tot_seconds)[source]

Compute reference epoch and seconds from VDIF standard to MJD and seconds. Using datetime.

epoch_six : int
epoch directly from VDIF header.
tot_seconds : int
seconds directly from VDIF header.
epoch
epoch MJD.
seconds
epoch seconds.
lib_vdif.write_samples(samples, bits_per_sample, word_size=32)[source]

Write vdif samples. Used by vdif signal generator.

lib_vdif.write_samples_raw(samples, bits_per_sample, word_size=32)[source]

Write vdif samples.

Untested. Limited to bits_per_samples that are divisor of 8.

lib_vdif.write_words_to_file(f, words)[source]
lib_vdif.write_words_to_file_raw(f, words)[source]