lib_ini_exper module¶
Routines for processing the experiment configuration.
-
lib_ini_exper.
check_errors_ini_exper
(data_dir, ini_folder, ini_stations, ini_sources, ini_delay_model, ini_media)[source]¶ Check experiment configuration for errors. Currently simply checking existence of paths and files.
- data_dir : str
- path to media folder.
- ini_folder : str
- path to experiment folder.
- ini_stations : str
- path to stations.ini (absolute).
- ini_sources : str
- path to sources.ini (absolute).
- ini_delay_model : str
- path to delay_model.ini (absolute).
- ini_media : str
- path to media.ini (absolute)
- init_success : int
- 0 if errors, 1 otherwise.
TO DO:Add more checks. Consider checking ini files syntax, range of values, etc.
-
lib_ini_exper.
gen_delays_ini_file
(file_stations_ini, params_array_stations, file_sources_ini, params_array_sources, file_delay_model_ini, params_array_delay_model, file_delays_ini, mjd_start, seconds_ref, tot_steps, step_seconds=1, seconds_offset=0, v=1)[source]¶ Generate file with delay polynomials from delay model.
- file_stations_ini : str
- path to stations.ini.
- params_array_stations : list
- configuration of stations.ini.
- file_sources_ini : str
- path to sources.ini.
- params_array_sources : list
- configuration of sources.ini.
- file_delay_model_ini : str
- path to delay_model.ini.
- params_array_delay_model : list
- configuration of delay_mode.ini.
- file_delays_ini : str
- path to delays.ini [will write].
- mjd_start : int
- MJD for the start of the scan.
- seconds_ref : int
- seconds in MJD for the start of the scan.
- tot_steps : int
- number of accumulation periods in the scan.
- step_seconds : str(float)
- accumulation period [s].
- seconds_offset
- 0
- v : int
- verbose if 1.
- s_delay
- None if error, ini configuration to write if successful.
-
lib_ini_exper.
get_num_partitions_red
(one_baseline_per_task, accumulation_time, signal_duration, stations, auto_stations, max_num_channels, max_num_pols, v, file_log)[source]¶ Get number of reducers based on the experiment configuration.
- one_baseline_per_task : int
- [default 0] 0 for all baselines per task, 1 for single baseline per task.
- accumulation_time : str(float)
- accumulation period in seconds.
- signal_duration : str(float)
- signal duration in seconds.
- stations : int
- number of stations.
- auto_stations : int
- 1 if compute autocorrelations, 0 otherwise.
- max_num_channels : int
- maximum number of channels per media file.
- max_num_pol : int
- maximum number of polarizations per media file.
- v : int
- verbose if 1.
- file_log : file handler
- handler to log file.
- total_partitions : int
- number of reducers.
- This function is critical (together with the custom partitioner) to parallelize the
- reduce phase and keep a well balanced load among the reducers.
-
lib_ini_exper.
process_ini_correlation
(params_array_corr, file_log, v=1)[source]¶ Process correlation.ini file.
- params_array_corr
- list with correlation configuration.
- file_log
- handler for log file.
- v
- verbose if 1.
- stations : int
- number of stations.
- fft_size : int
- number of coefficients in DFT.
- accumulation_time : str(float)
- accumulation time in seconds.
- windowing : str
- window type to be applied before DFT.
- mjd_start : int
- MJD for the start of the scan.
- seconds_start : int
- number of seconds in MJD for the start of the scan.
- seconds_duration : int
- duration of the scan in seconds.
- phase_calibration : int
- 1 if phase calibration tones will be extracted, 0 otherwise.
- auto_stations : int
- 1 if compute autocorrelations, 0 otherwise.
- auto_pols : int
- 1 if crosspolarizations (see msvf.py).
- first_frame_num : int
- [testing only] start reading after this frame id (-1 to read from first frame).
- num_frames : int
- [testing only] read this number of frames per second (-1 to read all frames)
- num_pols : int
- [testing only] default -1.
-
lib_ini_exper.
process_ini_files
(data_dir, ini_stations, ini_sources, ini_delay_model, ini_delays, ini_media, ini_correlation, one_baseline_per_task, v=1, file_log=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>)[source]¶ Process .ini files.
- data_dir : str
- path to folder with media files.
- ini_stations : str
- path to stations.ini.
- ini_sources : str
- path to sources.ini.
- ini_delay_model : str
- path to delay_model.ini.
- ini_delays : str
- path to delays.ini.
- ini_media : str
- path to media.ini.
- ini_correlation : str
- path to correlation.ini.
- one_baseline_per_task : int
- default 0.
- v : int
- verbose if 1.
- file_log : file handler
- handler for log file.
- stations_serial_str : str
- serialized version of stations.ini.
- media_serial_str : str
- serialized version of media.ini.
- correlation_serial_str : str
- serialized version of correlation.ini.
- delays_serial_str : str
- serialized version of delays.ini.
- auto_stations : int
- 1 if compute autocorrelations, 0 otherwise.
- auto_pols : int
- 1 if compute cross-polarization correlations, 2 if only same-polarization correlations (see msvf.py).
- fft_size : int
- requested number of coefficients in the FFT.
- accumulation_time : str
- integration period duration [s].
- stations : int
- number of stations (it will be used to compute number of mappers...).
- ref_epoch : int
- MJD for the start of the scan.
- signal_start : int
- seconds after MJD for the start of the scan.
- signal_duration : float
- duration of the scan [s].
- input_files : list of str
- filenames for media files.
- first_frame_num : int
- [default -1] This can be used to force the mapper to start reading frames after this id.
- num_frames : int
- [default -1] This can be used to force the mapper to read only this number of frames for each second.
- codecs_serial : str
- [default “0”] Serialized version of the codebook if using compression.
- max_packet_size : int
- number of bytes in the largest frame of the media files.
- total_frames : int
- number of frames in all the media files.
- total_partitions : int
- number of reducer tasks.
- windowing : str
- windowing as defined in correlation.ini.
- phase_calibration : int
- 1 if extract phase calibration tones, 0 otherwise.
- delay_error : None or str
- None if no errors during delay computations, str with error otherwise.
- error_str_v :
- list of str with errors for media files not existing.
- num_pols : int
- -1
TO DO:Remove num_pols.Remove vq.
-
lib_ini_exper.
process_ini_media
(params_array_media, data_dir, v, file_log)[source]¶ Process media.ini file.
- params_array_corr : list
- list with media configuration.
- data_dir : str
- folder with media files.
- v : int
- verbose if 1.
- file_log : file handler
- handler for log file.
- v_stations : list of str
- all stations ids corresponding to all the media files listed.
- max_num_channels : int
- maximum number of channels per media file.
- max_num_pols : int
- maximum number of polarizations per media file.
- input_files : list of str
- filenames for media files.
- total_frames : int
- total frames considering all media files.
- max_packet_size : int
- maximum frame size per media file.
- error_str_v : list of str
- errors on missing media files ([] if no errors).
Considerations:Currently assuming that all the media files have frames with the same size.Currently assuming that all the files listed are processed, and that there is a section for eachof the listed files.
-
lib_ini_exper.
process_ini_stations
(params_array_stations, v_stations)[source]¶ Process stations.ini file.
- params_array_corr
- list with stations configuration.
- v_stations
- list of str with all stations ids corresponding to all the media files listed (from media).
- v_id_stations
- list of int with the ids corresponding to v_stations.