template<typename _realT>
struct mx::improc::HCIobservation< _realT >

The basic high contrast imaging data type.

This class manages file reading, resizing, co-adding, pre-processing (masking and filtering), and final image combination.

Template Parameters

_realT is the floating point type in which to do all arithmetic.

Definition at line 118 of file HCIobservation.hpp.

#include <improc/HCIobservation.hpp>

Inheritance diagram for mx::improc::HCIobservation< _realT >:

Public Types
typedef _realT	realT
	The arithmetic type used for calculations. Does not have to match the type in images on disk.

typedef Eigen::Array< realT, Eigen::Dynamic, Eigen::Dynamic >	eigenImageT
	The Eigen image array type basted on realT.

Public Member Functions
int	readPSFSub (const std::string &dir, const std::string &prefix, const std::string &ext, size_t nReductions)
	Read in already PSF-subtracted files.

Construction and Initialization
	HCIobservation ()
	Default c'tor.

	HCIobservation (const std::string &dir, const std::string &prefix, const std::string &ext)
	Construct and load the target file list.

	HCIobservation (const std::string &fileListFile)
	Construct using a file containing the target file list.

	HCIobservation (const std::string &dir, const std::string &prefix, const std::string &ext, const std::string &RDIdir, const std::string &RDIprefix, const std::string &RDIext="")
	Construct and load the target file list and the RDI file list.

	HCIobservation (const std::string &fileListFile, const std::string &RDIfileListFile)
	Construct using a file containing the target file list and a file containing the RDI target file list.

void	load_fileList (const std::string &dir, const std::string &prefix, const std::string &ext)
	Load the file list.

void	load_fileList (const std::string &fileListFile)
	Load the file list from a file.

void	load_RDIfileList (const std::string &dir, const std::string &prefix, const std::string &ext)
	Load the RDI basis file list.

void	load_RDIfileList (const std::string &fileListFile)
	Load the RDI basis file list from a file.

Thresholding
Thresholds are applied to a list of files before it is read, based on the image qualities supplied.
int	threshold (std::vector< std::string > &fileList, const std::string &qualityFile, realT qualityThreshold)
	Read the image qualities from a qualityFile and apply the threshold to a fileList.

Public Attributes
The Input Target Images
eigenCube< realT >	m_tgtIms
	The target image cube.

int	m_Nims { 0 }
	Number of images in m_tgtIms.

int	m_Nrows { 0 }
	Number of rows of the images in m_tgtIms.

int	m_Ncols { 0 }
	Number of columns of the images in m_tgtIms.

int	m_Npix { 0 }
	Pixels per image, that is Nrows*Ncols.

std::vector< double >	m_imageMJD
	Vector of target image times, in MJD.

std::vector< fits::fitsHeader >	m_heads
	Vector of FITS headers, one per file, populated with the values for the keywords.

bool	m_filesRead { false }
	Whether or not the m_fileList has been read.

bool	m_filesDeleted { false }
	Whether or not the specified files have been deleted from m_fileList.

std::vector< realT >	m_comboWeights
	Vector to hold the image weights read from the m_weightFile.

The Input Reference Images
eigenCube< realT >	m_refIms
	The optional reference image cube.

std::vector< double >	m_RDIimageMJD
	Vector of reference image times, in MJD.

std::vector< fits::fitsHeader >	m_RDIheads
	Vector of FITS headers, one per reference file, populated with the values for the keywords.

bool	m_RDIfilesRead { false }
	Whether or not the reference files have been read.

bool	m_RDIfilesDeleted { false }
	Whether or not the specified files have been deleted from m_RDIfileList.

The Reduced Data
std::vector< eigenCube< realT > >	m_psfsub
	The PSF subtracted images.

eigenCube< realT >	m_finim
	The final combined images, one for each cube in psfsub.

Target File Reading
Options to control which files are read, how they are read, what meta data is extracted from FITS headers, sizing and masking, etc.
std::vector< std::string >	m_fileList
	The list of files to read in.

int	m_deleteFront { 0 }
	Specify how many files from m_fileList to delete from the front of the list.

int	m_deleteBack { 0 }
	Specify how many files from m_fileList to delete from the back of the list.

std::string	m_qualityFile
	File containing 2 space-delimited columns of fileVame qualityValue pairs.

realT	m_qualityThreshold { 0 }
	Threshold to apply to qualityValues read from qualityFile.

bool	m_thresholdOnly { false }
	Just prints the names and qualities of the files which pass threshold, and stop.

std::string	m_MJDKeyword { "DATE-OBS" }
	Name of the keyword to use for the image date.

bool	m_MJDisISO8601 { true }
	Whether or not the date is in ISO 8601 format.

realT	m_MJDUnits { 1.0 }
	If the date is not ISO 8601, this specifies the conversion to Julian Days (i.e. seconds to days)

std::vector< std::string >	m_keywords
	Vector of FITS header keywords to read from the files in m_fileList.

int	m_imSize { 0 }
	Set the image size. Images are cut down to this size after reading.

int	readFiles ()
	Read the list of files, cut to size, and preprocess.

virtual int	postReadFiles ()
	Perform post-read actions for the target images, for use by derived classes.

virtual int	postCoadd ()
	Perform post-coadd actions for the target images, for use by derived classes.

Reference File Reading
For RDI, Options to control which files are read, how they are read, what meta data is extracted from FITS headers, sizing and masking, etc.
std::vector< std::string >	m_RDIfileList
	The list of files to read in for the RDI basis.

int	m_RDIdeleteFront { 0 }
	Specify how many files from m_RDIfileList to delete from the front of the list.

int	m_RDIdeleteBack { 0 }
	Specify how many files from m_RDIfileList to delete from the back of the list.

std::string	m_RDIqualityFile
	File containing 2 space-delimited columns of fileMame qualityValue pairs for the reference images.

realT	m_RDIqualityThreshold { 0 }
	Threshold to apply to qualityValues read from qualityFile.

std::vector< std::string >	m_RDIkeywords
	Vector of FITS header keywords to read from the files in m_fileList.

int	readRDIFiles ()
	Read the list of reference files, cut to size, and preprocess.

virtual int	postRDIReadFiles ()
	Perform post-read actions for the RDI images, for use by derived classes.

virtual int	postRDICoadd ()
	Perform post-coadd actions, for use by derived classes.

Coadding
These parameters control whether and how the images are coadded after being read. Coadding can be done up to a given number of images, and/or a given elapsed time. Averages the values of given Keywords as well.
int	m_coaddCombineMethod { HCI::noCombine }
	Determine how to coadd the raw images.

int	m_coaddMaxImno { 0 }
	Maximum number of images to coadd.

realT	m_coaddMaxTime { 0 }
	Maximum elapsed time over which to coadd the images.

std::vector< std::string >	m_coaddKeywords
	The values of these keywords will be averaged and replaced.

void	coaddImages (int coaddCombineMethod, int coaddMaxImno, int coaddMaxTime, std::vector< std::string > &coaddKeywords, std::vector< double > &imageMJD, std::vector< fits::fitsHeader > &heads, eigenCube< realT > &ims)
	Coadd the images.

Masking
A 1/0 mask can be supplied, which is used in pre-processing and in image combination.
std::string	m_maskFile
	Specify a mask file to apply.

eigenImageT	m_mask
	The mask.

eigenCube< realT >	m_maskCube

void	readMask ()
	Read the mask file, resizing to imSize if needed.

virtual void	makeMaskCube ()
	Populate the mask cube which is used for post-processing.

Pre-Processing
These options control the pre-processing masking and filtering. They are performed in the following order: mask applied (enabled by m_preProcess_mask radial profile subtraction (enabled by m_preProcess_subradprof) mask applied (enabled by m_preProcess_mask symmetric median unsharp mask (m_preProcess_gaussUSM_fwhm) symmetric Gaussian unsharp mask (m_preProcess_gaussUSM_fwhm) mask applied (enabled by m_preProcess_mask azimuthal unsharp mask (m_preProcess_azUSM_azW, and m_preProcess_azUSM_radW) mask applied (enabled by m_preProcess_mask) mean subtraction (enabled by m_preProcess_meanSubMethod) mask applied (enabled by m_preProcess_mask) pixel time-series normalization (enabled by m_preProcess_pixelTSNormMethod)
bool	m_skipPreProcess { false }
	Don't do any of the pre-processing steps (including coadding).

bool	m_preProcess_beforeCoadd { false }
	controls whether pre-processing takes place before or after coadding

bool	m_preProcess_mask { true }
	If true, the mask is applied during each pre-processing step.

bool	m_preProcess_subradprof { false }
	If true, a radial profile is subtracted from each image.

realT	m_preProcess_azUSM_azW { 0 }
	Azimuthal boxcar width for azimuthal unsharp mask [pixels].

realT	m_preProcess_azUSM_maxAz { 45 }
	Mazimum azimuthal boxcar width for azimuthal unsharp mask [degrees].

realT	m_preProcess_azUSM_radW { 0 }
	Radial boxcar width for azimuthal unsharp mask [pixels].

int	m_preProcess_medianUSM_fwhm { 0 }
	Kernel FWHM for symmetric box median unsharp mask (USM)

realT	m_preProcess_gaussUSM_fwhm { 0 }
	Kernel FWHM for symmetric Gaussian unsharp mask (USM)

HCI::meanSubMethod	m_preProcess_meanSubMethod { HCI::meanSubMethod::none }
	The mean subtraction method during pre-processing.

HCI::pixelTSNormMethod	m_preProcess_pixelTSNormMethod { HCI::pixelTSNormMethod::none }
	Specify if each pixel time-series is normalized.

realT	m_pixelTSSigma { 3 }
	Sigma-clipping parameter for pixel time-series normalization.

std::string	m_preProcess_outputPrefix
	Set path and file prefix to output the pre-processed images.

bool	m_preProcess_only { false }
	If true, then we stop after pre-processing.

void	preProcess (eigenCube< realT > &ims)
	Do the pre-processing.

void	preProcess_meanSub (eigenCube< realT > &ims)
	Do mean subtraction as part of pre-processing.

void	preProcess_pixelTSNorm (eigenCube< realT > &ims)
	Do pixel time-series normalization as part of pre-processing.

Image Combination
These options control how the final image combination is performed.
int	m_combineMethod { HCI::meanCombine }
	Determine how to combine the PSF subtracted images.

std::string	m_weightFile
	Specifies a file containing the image weights, for combining with weighted mean.

realT	m_sigmaThreshold { 0 }
	The standard deviation threshold used if combineMethod == HCI::sigmaMeanCombine.

realT	m_minGoodFract { 0.0 }

int	readWeights ()
	Read the image weights from m_weightFile.

void	combineFinim ()
	Combine the images into a single final image.

Output
These options control the ouput of the final combined images and the individual PSF subtracted images.
std::string	m_auxDataDir { "/tmp/klipReduceAux/" }
	Location for temporary auxilliary output files (e.g. masks)

bool	m_moveAuxDataDir { true }
	Whether or not to move the temp. aux files.

int	m_doWriteFinim { 1 }
	Set whether the final combined image is written to disk.

std::string	m_outputDir
	The directory where to write output files.

std::string	m_finimName { "finim_" }
	The base file name of the output final image.

bool	m_exactFinimName { false }
	Use m_finimName exactly as specified, without appending a number or an extension.

bool	m_doOutputPSFSub { false }
	Controls whether or not the individual PSF subtracted images are written to disk.

std::string	m_PSFSubPrefix

void	outputPreProcessed ()
	Output the pre-processed target images.

void	outputRDIPreProcessed ()
	Output the pre-processed reference images.

void	stdFitsHeader (fits::fitsHeader &head)
	Fill in the HCIobservation standard FITS header.

void	writeFinim (fits::fitsHeader *addHead=0)
	Write the final combined image to disk.

void	outputPSFSub (fits::fitsHeader *addHead=0)
	Write the PSF subtracted images to disk.

Member Typedef Documentation

◆ eigenImageT

template<typename _realT >

typedef Eigen::Array<realT, Eigen::Dynamic, Eigen::Dynamic> mx::improc::HCIobservation< _realT >::eigenImageT

The Eigen image array type basted on realT.

Definition at line 125 of file HCIobservation.hpp.

◆ realT

template<typename _realT >

typedef _realT mx::improc::HCIobservation< _realT >::realT

The arithmetic type used for calculations. Does not have to match the type in images on disk.

Definition at line 122 of file HCIobservation.hpp.

Constructor & Destructor Documentation

◆ HCIobservation() [1/5]

template<typename _realT >

mx::improc::HCIobservation< _realT >::HCIobservation ( )

Default c'tor.

Definition at line 726 of file HCIobservation.hpp.

◆ HCIobservation() [2/5]

template<typename _realT >

mx::improc::HCIobservation< _realT >::HCIobservation	(	const std::string &	dir,
		const std::string &	prefix,
		const std::string &	ext
	)

Construct and load the target file list.

Populates the m_fileList vector by searching on disk for files which match "dir/prefix*.ext". See load_fileList

Parameters

[in]	dir	the directory to search.
[in]	prefix	the initial part of the file name. Can be empty "".
[in]	ext	the extension to append to the file name, must include the '.'.

Definition at line 731 of file HCIobservation.hpp.

◆ HCIobservation() [3/5]

template<typename _realT >

mx::improc::HCIobservation< _realT >::HCIobservation ( const std::string & fileListFile )

explicit

Construct using a file containing the target file list.

Populates the m_fileList vector by reading the file, which should be a single column of new-line delimited file names.

Parameters

[in] fileListFile a file name path to read.

Definition at line 737 of file HCIobservation.hpp.

◆ HCIobservation() [4/5]

template<typename _realT >

mx::improc::HCIobservation< _realT >::HCIobservation	(	const std::string &	dir,
		const std::string &	prefix,
		const std::string &	ext,
		const std::string &	RDIdir,
		const std::string &	RDIprefix,
		const std::string &	RDIext = `""`
	)

Construct and load the target file list and the RDI file list.

Populates the m_fileList vector by searching on disk for files which match "dir/prefix*.ext". See load_fileList

Populates the m_RDIfileList vector by searching on disk for files which match "RDIdir/RDIprefix*.RDIext". See load_RDIfileList

Parameters

[in]	dir	the directory to search.
[in]	prefix	the initial part of the file name. Can be empty "".
[in]	ext	the extension to append to the file name, must include the '.'.
[in]	RDIdir	the directory to search for the reference files.
[in]	RDIprefix	the initial part of the file name for the reference files. Can be empty "".
[in]	RDIext	[optional] the extension to append to the RDI file name, must include the '.'. If empty "" then same extension as target files is used.

Definition at line 743 of file HCIobservation.hpp.

◆ HCIobservation() [5/5]

template<typename _realT >

mx::improc::HCIobservation< _realT >::HCIobservation	(	const std::string &	fileListFile,
		const std::string &	RDIfileListFile
	)

Construct using a file containing the target file list and a file containing the RDI target file list.

Populates the m_fileList vector by reading the file, which should be a single column of new-line delimited file names.

Populates the m_RDIfileList vector by reading the file, which should be a single column of new-line delimited file names.

Parameters

[in]	fileListFile	a file name path to read for the target file names.
[in]	RDIfileListFile	a file name path to read for the reference file names.

Definition at line 764 of file HCIobservation.hpp.

Member Function Documentation

◆ coaddImages()

template<typename _realT >

void mx::improc::HCIobservation< _realT >::coaddImages	(	int	coaddCombineMethod,
		int	coaddMaxImno,
		int	coaddMaxTime,
		std::vector< std::string > &	coaddKeywords,
		std::vector< double > &	imageMJD,
		std::vector< fits::fitsHeader > &	heads,
		eigenCube< realT > &	ims
	)

Coadd the images.

Todo:: this should really include end of exposure too.

Todo:: this needs to handle averaging of angles

Definition at line 1289 of file HCIobservation.hpp.

References mx::sys::get_curr_time(), mx::improc::eigenCube< dataT >::image(), mx::sys::ISO8601DateTimeStrMJD(), mx::improc::eigenCube< dataT >::mean(), mx::improc::HCI::meanCombine, mx::improc::eigenCube< dataT >::median(), mx::improc::HCI::medianCombine, mx::improc::HCI::noCombine, and mx::ioutils::pathFilename().

◆ combineFinim()

template<typename _realT >

void mx::improc::HCIobservation< _realT >::combineFinim ( )

Combine the images into a single final image.

Images are combined by the method specified in combineMethod

Definition at line 1868 of file HCIobservation.hpp.

References mx::sys::get_curr_time(), mx::improc::HCI::meanCombine, mx::improc::HCI::medianCombine, mx::improc::HCI::noCombine, and mx::improc::HCI::sigmaMeanCombine.

◆ load_fileList() [1/2]

template<typename _realT >

void mx::improc::HCIobservation< _realT >::load_fileList	(	const std::string &	dir,
		const std::string &	prefix,
		const std::string &	ext
	)

Load the file list.

Populates the m_fileList vector by searching on disk for files which match the given parameters. Uses mx::getFileNames to search for all files which match "dir/prefix*.ext".

Parameters

[in]	dir	the directory to search.
[in]	prefix	the initial part of the file name. Can be empty "".
[in]	ext	the extension to append to the file name, which must include the '.'. Can be empty "".

Definition at line 771 of file HCIobservation.hpp.

References mx::ioutils::getFileNames().

◆ load_fileList() [2/2]

template<typename _realT >

void mx::improc::HCIobservation< _realT >::load_fileList ( const std::string & fileListFile )

Load the file list from a file.

Populates the m_fileList vector by reading the file, which should be a single column of new-line delimited file names.

Parameters

[in] fileListFile a file name path to read.

Definition at line 779 of file HCIobservation.hpp.

References mx::ioutils::readColumns().

◆ load_RDIfileList() [1/2]

template<typename _realT >

void mx::improc::HCIobservation< _realT >::load_RDIfileList	(	const std::string &	dir,
		const std::string &	prefix,
		const std::string &	ext
	)

Load the RDI basis file list.

Populates the m_RDIfileList vector by searching on disk for files which match the given parameters. Uses mx::getFileNames to search for all files which match "dir/prefix*.ext".

Parameters

[in]	dir	the directory to search.
[in]	prefix	the initial part of the file name. Can be empty "".
[in]	ext	the extension to append to the file name, which must include the '.'. Can be empty "".

Definition at line 786 of file HCIobservation.hpp.

References mx::ioutils::getFileNames().

◆ load_RDIfileList() [2/2]

template<typename _realT >

void mx::improc::HCIobservation< _realT >::load_RDIfileList ( const std::string & fileListFile )

Load the RDI basis file list from a file.

Populates the m_fileList vector by reading the file, which should be a single column of new-line delimited file names.

Parameters

[in] fileListFile a file name path to read.

Definition at line 795 of file HCIobservation.hpp.

References mx::ioutils::readColumns().

◆ makeMaskCube()

template<typename realT >

void mx::improc::HCIobservation< realT >::makeMaskCube ( )

virtual

Populate the mask cube which is used for post-processing.

Derived classes can do this as appropriate, e.g. by rotating the mask.

Exceptions

mx::err::invalidconfig if mask is not the same size as the images

Reimplemented in mx::improc::ADIobservation< _realT, _derotFunctObj >.

Definition at line 1525 of file HCIobservation.hpp.

◆ outputPreProcessed()

template<typename _realT >

void mx::improc::HCIobservation< _realT >::outputPreProcessed ( )

Output the pre-processed target images.

Todo:: Should add a HISTORY card here

Definition at line 1979 of file HCIobservation.hpp.

References mx::ioutils::createDirectories(), mx::ioutils::parentPath(), and mx::fits::fitsFile< dataT >::write().

◆ outputPSFSub()

template<typename _realT >

void mx::improc::HCIobservation< _realT >::outputPSFSub ( fits::fitsHeader * addHead = 0 )

Write the PSF subtracted images to disk.

Definition at line 2105 of file HCIobservation.hpp.

References mx::fits::fitsHeader::append(), mx::fits::fitsHeaderGitStatus(), and mx::fits::fitsFile< dataT >::write().

◆ outputRDIPreProcessed()

template<typename _realT >

void mx::improc::HCIobservation< _realT >::outputRDIPreProcessed ( )

Output the pre-processed reference images.

◆ postCoadd()

template<typename _realT >

int mx::improc::HCIobservation< _realT >::postCoadd ( )

virtual

Perform post-coadd actions for the target images, for use by derived classes.

Returns: 0 on success; <0 on error.

Reimplemented in mx::improc::ADIobservation< _realT, _derotFunctObj >.

Definition at line 1046 of file HCIobservation.hpp.

◆ postRDICoadd()

template<typename _realT >

int mx::improc::HCIobservation< _realT >::postRDICoadd ( )

virtual

Perform post-coadd actions, for use by derived classes.

A key example is to update keywords after the averaging occurs in coaddImages().

Returns: 0 on success; <0 on error.

Reimplemented in mx::improc::ADIobservation< _realT, _derotFunctObj >.

Definition at line 1237 of file HCIobservation.hpp.

◆ postRDIReadFiles()

template<typename _realT >

int mx::improc::HCIobservation< _realT >::postRDIReadFiles ( )

virtual

Perform post-read actions for the RDI images, for use by derived classes.

Reimplemented in mx::improc::ADIobservation< _realT, _derotFunctObj >.

Definition at line 1231 of file HCIobservation.hpp.

◆ postReadFiles()

template<typename _realT >

int mx::improc::HCIobservation< _realT >::postReadFiles ( )

virtual

Perform post-read actions for the target images, for use by derived classes.

Reimplemented in mx::improc::ADIobservation< _realT, _derotFunctObj >.

Definition at line 1040 of file HCIobservation.hpp.

◆ preProcess()

template<typename _realT >

void mx::improc::HCIobservation< _realT >::preProcess ( eigenCube< realT > & ims )

Do the pre-processing.

Parameters

[in] ims the image cube, should be either m_tgtIms or m_refIms

Definition at line 1549 of file HCIobservation.hpp.

References mx::improc::filterImage(), mx::sys::get_curr_time(), mx::improc::eigenCube< dataT >::image(), mx::ipc::ompLoopWatcher< _outputT, _printPretty, _printLoops, _printPercent, _printNLine, _time >::incrementAndOutputStatus(), mx::improc::medianSmooth(), mx::improc::radprofim(), and mx::improc::zeroNaNs().

◆ preProcess_meanSub()

template<typename _realT >

void mx::improc::HCIobservation< _realT >::preProcess_meanSub ( eigenCube< realT > & ims )

Do mean subtraction as part of pre-processing.

Parameters

[in] ims the image cube, should be either m_tgtIms or m_refIms

Definition at line 1714 of file HCIobservation.hpp.

References mx::improc::eigenCube< dataT >::image(), mx::improc::eigenCube< dataT >::mean(), mx::improc::HCI::meanImage, mx::improc::eigenCube< dataT >::median(), mx::improc::HCI::medianImage, and mx::improc::HCI::none.

◆ preProcess_pixelTSNorm()

template<typename _realT >

void mx::improc::HCIobservation< _realT >::preProcess_pixelTSNorm ( eigenCube< realT > & ims )

Do pixel time-series normalization as part of pre-processing.

Parameters

[in] ims the image cube, should be either m_tgtIms or m_refIms

Definition at line 1752 of file HCIobservation.hpp.

References mx::improc::eigenCube< dataT >::image(), mx::improc::HCI::none, mx::improc::HCI::rmsSigmaClipped, mx::improc::HCI::unknown, and mx::math::vectorVariance().

◆ readFiles()

template<typename realT >

int mx::improc::HCIobservation< realT >::readFiles ( )

Read the list of files, cut to size, and preprocess.

Returns: 0 on success, -1 on error.

Definition at line 804 of file HCIobservation.hpp.

References mx::fits::fitsHeader::append(), mx::fits::fitsHeader::count(), mx::sys::get_curr_time(), mx::sys::ISO8601date2mjd(), mx::improc::HCI::noCombine, mx::fits::fitsFile< dataT >::read(), mx::fits::fitsFile< dataT >::setReadSize(), and mx::improc::zeroNaNCube().

◆ readMask()

template<typename _realT >

void mx::improc::HCIobservation< _realT >::readMask ( )

Read the mask file, resizing to imSize if needed.

Todo:: here re-size mask if needed to match imSize

Definition at line 1501 of file HCIobservation.hpp.

References mx::fits::fitsFile< dataT >::read().

◆ readPSFSub()

template<typename _realT >

int mx::improc::HCIobservation< _realT >::readPSFSub	(	const std::string &	dir,
		const std::string &	prefix,
		const std::string &	ext,
		size_t	nReductions
	)

Read in already PSF-subtracted files.

Used to take up final processing after applying some non-klipReduce processing steps to PSF-subtracted images.

Definition at line 2172 of file HCIobservation.hpp.

References mx::fits::fitsHeader::append(), mx::improc::HCI::combineMethodFmStr(), mx::fits::fitsHeader::count(), mx::sys::get_curr_time(), mx::ioutils::getFileNames(), mx::sys::ISO8601date2mjd(), mx::fits::fitsFile< dataT >::read(), mx::ioutils::readColumns(), mx::fits::fitsFile< dataT >::setReadSize(), and mx::improc::zeroNaNCube().

◆ readRDIFiles()

template<typename _realT >

int mx::improc::HCIobservation< _realT >::readRDIFiles ( )

Read the list of reference files, cut to size, and preprocess.

The target files must be read with readFiles() before calling this method.

Returns: 0 on success, -1 on error.

Definition at line 1053 of file HCIobservation.hpp.

References mx::fits::fitsHeader::append(), mx::sys::get_curr_time(), mx::sys::ISO8601date2mjd(), mx::improc::HCI::noCombine, mx::fits::fitsFile< dataT >::read(), mx::fits::fitsFile< dataT >::setReadSize(), and mx::improc::zeroNaNCube().

◆ readWeights()

template<typename _realT >

int mx::improc::HCIobservation< _realT >::readWeights ( )

Read the image weights from m_weightFile.

This is called by readFiles().

Returns: 0 on success, -1 on error.

Definition at line 1811 of file HCIobservation.hpp.

References mx::ioutils::pathFilename(), and mx::ioutils::readColumns().

◆ stdFitsHeader()

template<typename _realT >

void mx::improc::HCIobservation< _realT >::stdFitsHeader ( fits::fitsHeader & head )

Fill in the HCIobservation standard FITS header.

Parameters

head	[in.out] the fistHeader structure which will have cards appended to it.

Definition at line 2007 of file HCIobservation.hpp.

References mx::fits::fitsHeader::append(), mx::improc::HCI::combineMethodStr(), mx::improc::HCI::noCombine, and mx::ioutils::pathFilename().

◆ threshold()

template<typename _realT >

int mx::improc::HCIobservation< _realT >::threshold	(	std::vector< std::string > &	fileList,
		const std::string &	qualityFile,
		realT	qualityThreshold
	)

Read the image qualities from a qualityFile and apply the threshold to a fileList.

This is called by readFiles().

Returns: 0 on success, -1 on error.

Parameters

	fileList	[in.out] the fileList to threshold
[in]	qualityFile	the path to the file containing qualities, one per file.
[in]	qualityThreshold	the quality threshold to apply

Definition at line 1243 of file HCIobservation.hpp.

References mx::ioutils::pathFilename(), and mx::ioutils::readColumns().

◆ writeFinim()

template<typename _realT >

void mx::improc::HCIobservation< _realT >::writeFinim ( fits::fitsHeader * addHead = 0 )

Write the final combined image to disk.

Definition at line 2059 of file HCIobservation.hpp.

References mx::fits::fitsHeader::append(), mx::improc::HCI::combineMethodStr(), mx::fits::fitsHeaderGitStatus(), mx::ioutils::getSequentialFilename(), mx::improc::HCI::sigmaMeanCombine, and mx::fits::fitsFile< dataT >::write().

Member Data Documentation

◆ m_auxDataDir

template<typename _realT >

std::string mx::improc::HCIobservation< _realT >::m_auxDataDir { "/tmp/klipReduceAux/" }

Location for temporary auxilliary output files (e.g. masks)

Definition at line 637 of file HCIobservation.hpp.

◆ m_coaddCombineMethod

template<typename _realT >

int mx::improc::HCIobservation< _realT >::m_coaddCombineMethod { HCI::noCombine }

Determine how to coadd the raw images.

Possibilities are

HCI::noCombine – [default] do not combine. This turns off coadding.
HCI::medianCombine – coadded image is the median
HCI::meanCombine – coadded image is the simple mean

No other types of combination are currently supported for coadding.

Definition at line 452 of file HCIobservation.hpp.

◆ m_coaddKeywords

template<typename _realT >

std::vector<std::string> mx::improc::HCIobservation< _realT >::m_coaddKeywords

The values of these keywords will be averaged and replaced.

Definition at line 461 of file HCIobservation.hpp.

◆ m_coaddMaxImno

template<typename _realT >

int mx::improc::HCIobservation< _realT >::m_coaddMaxImno { 0 }

Maximum number of images to coadd.

Definition at line 455 of file HCIobservation.hpp.

◆ m_coaddMaxTime

template<typename _realT >

realT mx::improc::HCIobservation< _realT >::m_coaddMaxTime { 0 }

Maximum elapsed time over which to coadd the images.

Definition at line 458 of file HCIobservation.hpp.

◆ m_combineMethod

template<typename _realT >

int mx::improc::HCIobservation< _realT >::m_combineMethod { HCI::meanCombine }

Determine how to combine the PSF subtracted images.

Possibilities are

HCI::noCombine – do not combine
HCI::medianCombine – [default] final image is the median
HCI::meanCombine – final image is the simple mean
HCI::weightedMeanCombine – final image is the weighted mean. m_weightFile must be provided.
HCI::sigmaMeanCombine – final image is sigma clipped mean. If m_sigmaThreshold <= 0, then it reverts to meanCombine.

Definition at line 601 of file HCIobservation.hpp.

◆ m_comboWeights

template<typename _realT >

std::vector<realT> mx::improc::HCIobservation< _realT >::m_comboWeights

Vector to hold the image weights read from the m_weightFile.

After readWeights is executed by readFiles, this will contain the normalized weights.

Todo:: check how comboWeights are handled in coadding

Definition at line 245 of file HCIobservation.hpp.

◆ m_deleteBack

template<typename _realT >

int mx::improc::HCIobservation< _realT >::m_deleteBack { 0 }

Specify how many files from m_fileList to delete from the back of the list.

Definition at line 299 of file HCIobservation.hpp.

◆ m_deleteFront

template<typename _realT >

int mx::improc::HCIobservation< _realT >::m_deleteFront { 0 }

Specify how many files from m_fileList to delete from the front of the list.

Definition at line 296 of file HCIobservation.hpp.

◆ m_doOutputPSFSub

template<typename _realT >

bool mx::improc::HCIobservation< _realT >::m_doOutputPSFSub { false }

Controls whether or not the individual PSF subtracted images are written to disk.

true – write to disk
false – [default] don't write to disk

Definition at line 664 of file HCIobservation.hpp.

◆ m_doWriteFinim

template<typename _realT >

int mx::improc::HCIobservation< _realT >::m_doWriteFinim { 1 }

Set whether the final combined image is written to disk.

Definition at line 643 of file HCIobservation.hpp.

◆ m_exactFinimName

template<typename _realT >

bool mx::improc::HCIobservation< _realT >::m_exactFinimName { false }

Use m_finimName exactly as specified, without appending a number or an extension.

Output is still FITS format, regardless of extension. This will overwrite an existing file without asking.

Definition at line 658 of file HCIobservation.hpp.

◆ m_fileList

template<typename _realT >

std::vector<std::string> mx::improc::HCIobservation< _realT >::m_fileList

The list of files to read in.

This can be set on construction or by calling load_fileList

Definition at line 293 of file HCIobservation.hpp.

◆ m_filesDeleted

template<typename _realT >

bool mx::improc::HCIobservation< _realT >::m_filesDeleted { false }

Whether or not the specified files have been deleted from m_fileList.

Definition at line 239 of file HCIobservation.hpp.

◆ m_filesRead

template<typename _realT >

bool mx::improc::HCIobservation< _realT >::m_filesRead { false }

Whether or not the m_fileList has been read.

Definition at line 236 of file HCIobservation.hpp.

◆ m_finim

template<typename _realT >

eigenCube<realT> mx::improc::HCIobservation< _realT >::m_finim

The final combined images, one for each cube in psfsub.

Definition at line 280 of file HCIobservation.hpp.

◆ m_finimName

template<typename _realT >

std::string mx::improc::HCIobservation< _realT >::m_finimName { "finim_" }

The base file name of the output final image.

The complete name is formed by combining with a sequential number and the ".fits" extension. that is: m_finimName0000.fits. This behavior can be modified with m_exactFinimName.

Definition at line 652 of file HCIobservation.hpp.

◆ m_heads

template<typename _realT >

std::vector<fits::fitsHeader> mx::improc::HCIobservation< _realT >::m_heads

Vector of FITS headers, one per file, populated with the values for the keywords.

Definition at line 233 of file HCIobservation.hpp.

◆ m_imageMJD

template<typename _realT >

std::vector<double> mx::improc::HCIobservation< _realT >::m_imageMJD

Vector of target image times, in MJD.

Definition at line 230 of file HCIobservation.hpp.

◆ m_imSize

template<typename _realT >

int mx::improc::HCIobservation< _realT >::m_imSize { 0 }

Set the image size. Images are cut down to this size after reading.

Set to <= 0 to use images uncut.

Image sizes are not increased if this is larger than their size on disk.

Definition at line 344 of file HCIobservation.hpp.

◆ m_keywords

template<typename _realT >

std::vector<std::string> mx::improc::HCIobservation< _realT >::m_keywords

Vector of FITS header keywords to read from the files in m_fileList.

Definition at line 337 of file HCIobservation.hpp.

◆ m_mask

template<typename _realT >

eigenImageT mx::improc::HCIobservation< _realT >::m_mask

The mask.

Definition at line 484 of file HCIobservation.hpp.

◆ m_maskCube

template<typename _realT >

eigenCube<realT> mx::improc::HCIobservation< _realT >::m_maskCube

A cube of masks, one for each input image, which may be modified versions (e.g. rotated) of mask.

Definition at line 486 of file HCIobservation.hpp.

◆ m_maskFile

template<typename _realT >

std::string mx::improc::HCIobservation< _realT >::m_maskFile

Specify a mask file to apply.

No mask is applied if this is empty.

Definition at line 482 of file HCIobservation.hpp.

◆ m_minGoodFract

template<typename _realT >

realT mx::improc::HCIobservation< _realT >::m_minGoodFract { 0.0 }

The minimum fraction of good (un-masked) pixels to include in the final combination (0.0 to 1.0). If not met, then the pixel will be NaN-ed.

Definition at line 615 of file HCIobservation.hpp.

◆ m_MJDisISO8601

template<typename _realT >

bool mx::improc::HCIobservation< _realT >::m_MJDisISO8601 { true }

Whether or not the date is in ISO 8601 format.

Definition at line 331 of file HCIobservation.hpp.

◆ m_MJDKeyword

template<typename _realT >

std::string mx::improc::HCIobservation< _realT >::m_MJDKeyword { "DATE-OBS" }

Name of the keyword to use for the image date.

Specifies the keyword corresponding to the date. This is the "DATE" keyword for file write time, and usually "DATE-OBS" for actual observation time.

Default is "DATE-OBS".

If empty "", then image date is not read.

Definition at line 328 of file HCIobservation.hpp.

◆ m_MJDUnits

template<typename _realT >

realT mx::improc::HCIobservation< _realT >::m_MJDUnits { 1.0 }

If the date is not ISO 8601, this specifies the conversion to Julian Days (i.e. seconds to days)

Definition at line 334 of file HCIobservation.hpp.

◆ m_moveAuxDataDir

template<typename _realT >

bool mx::improc::HCIobservation< _realT >::m_moveAuxDataDir { true }

Whether or not to move the temp. aux files.

Definition at line 640 of file HCIobservation.hpp.

◆ m_Ncols

template<typename _realT >

int mx::improc::HCIobservation< _realT >::m_Ncols { 0 }

Number of columns of the images in m_tgtIms.

Definition at line 226 of file HCIobservation.hpp.

◆ m_Nims

template<typename _realT >

int mx::improc::HCIobservation< _realT >::m_Nims { 0 }

Number of images in m_tgtIms.

Definition at line 224 of file HCIobservation.hpp.

◆ m_Npix

template<typename _realT >

int mx::improc::HCIobservation< _realT >::m_Npix { 0 }

Pixels per image, that is Nrows*Ncols.

Definition at line 227 of file HCIobservation.hpp.

◆ m_Nrows

template<typename _realT >

int mx::improc::HCIobservation< _realT >::m_Nrows { 0 }

Number of rows of the images in m_tgtIms.

Definition at line 225 of file HCIobservation.hpp.

◆ m_outputDir

template<typename _realT >

std::string mx::improc::HCIobservation< _realT >::m_outputDir

The directory where to write output files.

Definition at line 646 of file HCIobservation.hpp.

◆ m_pixelTSSigma

template<typename _realT >

realT mx::improc::HCIobservation< _realT >::m_pixelTSSigma { 3 }

Sigma-clipping parameter for pixel time-series normalization.

Definition at line 565 of file HCIobservation.hpp.

◆ m_preProcess_azUSM_azW

template<typename _realT >

realT mx::improc::HCIobservation< _realT >::m_preProcess_azUSM_azW { 0 }

Azimuthal boxcar width for azimuthal unsharp mask [pixels].

If this is 0 then azimuthal-USM is not performed.

Definition at line 529 of file HCIobservation.hpp.

◆ m_preProcess_azUSM_maxAz

template<typename _realT >

realT mx::improc::HCIobservation< _realT >::m_preProcess_azUSM_maxAz { 45 }

Mazimum azimuthal boxcar width for azimuthal unsharp mask [degrees].

Limits width close to center, preventing wrap-around. Default is 45 degrees. Set to 0 for no maximum.

Definition at line 534 of file HCIobservation.hpp.

◆ m_preProcess_azUSM_radW

template<typename _realT >

realT mx::improc::HCIobservation< _realT >::m_preProcess_azUSM_radW { 0 }

Radial boxcar width for azimuthal unsharp mask [pixels].

If this is 0 then azimuthal-USM is not performed.

Definition at line 539 of file HCIobservation.hpp.

◆ m_preProcess_beforeCoadd

template<typename _realT >

bool mx::improc::HCIobservation< _realT >::m_preProcess_beforeCoadd { false }

controls whether pre-processing takes place before or after coadding

Definition at line 520 of file HCIobservation.hpp.

◆ m_preProcess_gaussUSM_fwhm

template<typename _realT >

realT mx::improc::HCIobservation< _realT >::m_preProcess_gaussUSM_fwhm { 0 }

Kernel FWHM for symmetric Gaussian unsharp mask (USM)

USM is not performed if this is 0.

Definition at line 549 of file HCIobservation.hpp.

◆ m_preProcess_mask

template<typename _realT >

bool mx::improc::HCIobservation< _realT >::m_preProcess_mask { true }

If true, the mask is applied during each pre-processing step.

Definition at line 522 of file HCIobservation.hpp.

◆ m_preProcess_meanSubMethod

template<typename _realT >

HCI::meanSubMethod mx::improc::HCIobservation< _realT >::m_preProcess_meanSubMethod { HCI::meanSubMethod::none }

The mean subtraction method during pre-processing.

Can only be none, meanImage, or meadianImage

Definition at line 554 of file HCIobservation.hpp.

◆ m_preProcess_medianUSM_fwhm

template<typename _realT >

int mx::improc::HCIobservation< _realT >::m_preProcess_medianUSM_fwhm { 0 }

Kernel FWHM for symmetric box median unsharp mask (USM)

USM is not performed if this is 0.

Definition at line 544 of file HCIobservation.hpp.

◆ m_preProcess_only

template<typename _realT >

bool mx::improc::HCIobservation< _realT >::m_preProcess_only { false }

If true, then we stop after pre-processing.

Definition at line 573 of file HCIobservation.hpp.

◆ m_preProcess_outputPrefix

template<typename _realT >

std::string mx::improc::HCIobservation< _realT >::m_preProcess_outputPrefix

Set path and file prefix to output the pre-processed images.

If empty, then pre-processed images are not output.

Definition at line 570 of file HCIobservation.hpp.

◆ m_preProcess_pixelTSNormMethod

template<typename _realT >

HCI::pixelTSNormMethod mx::improc::HCIobservation< _realT >::m_preProcess_pixelTSNormMethod { HCI::pixelTSNormMethod::none }

Specify if each pixel time-series is normalized.

This normalizaton is applied after centering. Can have the following values:

HCI::pixelTSNormMethod::none: no normalization (the default)
HCI::pixelTSNormMethod::rms: divide by the time-series rms
HCI::pixelTSNormMethod::rmsSigmaClipped: divide by the sigma-slipped time-series rms. The sigma is provided by m_preProcess_pixelTSSigma.

Definition at line 563 of file HCIobservation.hpp.

◆ m_preProcess_subradprof

template<typename _realT >

bool mx::improc::HCIobservation< _realT >::m_preProcess_subradprof { false }

If true, a radial profile is subtracted from each image.

Definition at line 524 of file HCIobservation.hpp.

◆ m_psfsub

template<typename _realT >

std::vector<eigenCube<realT> > mx::improc::HCIobservation< _realT >::m_psfsub

The PSF subtracted images.

This is a vector of cubes so that it can contain results from different reductions, e.g. different modes when using KLIP.

Definition at line 277 of file HCIobservation.hpp.

◆ m_PSFSubPrefix

template<typename _realT >

std::string mx::improc::HCIobservation< _realT >::m_PSFSubPrefix

Prefix of the FITS file names used to write individual PSF subtracted images to disk if m_doOutputPSFSub is true.

Definition at line 668 of file HCIobservation.hpp.

◆ m_qualityFile

template<typename _realT >

std::string mx::improc::HCIobservation< _realT >::m_qualityFile

File containing 2 space-delimited columns of fileVame qualityValue pairs.

If this is not empty and qualityThreshold is > 0, then only images where qualityValue >= qualityThreshold are read and processed.

The only restriction on qualityThreshold is that it is > 0. It is intendend to be something like Strehl ratio.

Definition at line 308 of file HCIobservation.hpp.

◆ m_qualityThreshold

template<typename _realT >

realT mx::improc::HCIobservation< _realT >::m_qualityThreshold { 0 }

Threshold to apply to qualityValues read from qualityFile.

If <= 0, then thresholding is not performed.

Definition at line 313 of file HCIobservation.hpp.

◆ m_RDIdeleteBack

template<typename _realT >

int mx::improc::HCIobservation< _realT >::m_RDIdeleteBack { 0 }

Specify how many files from m_RDIfileList to delete from the back of the list.

Definition at line 379 of file HCIobservation.hpp.

◆ m_RDIdeleteFront

template<typename _realT >

int mx::improc::HCIobservation< _realT >::m_RDIdeleteFront { 0 }

Specify how many files from m_RDIfileList to delete from the front of the list.

Definition at line 376 of file HCIobservation.hpp.

◆ m_RDIfileList

template<typename _realT >

std::vector<std::string> mx::improc::HCIobservation< _realT >::m_RDIfileList

The list of files to read in for the RDI basis.

This is set by calling load_RDIfileList

Definition at line 373 of file HCIobservation.hpp.

◆ m_RDIfilesDeleted

template<typename _realT >

bool mx::improc::HCIobservation< _realT >::m_RDIfilesDeleted { false }

Whether or not the specified files have been deleted from m_RDIfileList.

Definition at line 266 of file HCIobservation.hpp.

◆ m_RDIfilesRead

template<typename _realT >

bool mx::improc::HCIobservation< _realT >::m_RDIfilesRead { false }

Whether or not the reference files have been read.

Definition at line 263 of file HCIobservation.hpp.

◆ m_RDIheads

template<typename _realT >

std::vector<fits::fitsHeader> mx::improc::HCIobservation< _realT >::m_RDIheads

Vector of FITS headers, one per reference file, populated with the values for the keywords.

Definition at line 260 of file HCIobservation.hpp.

◆ m_RDIimageMJD

template<typename _realT >

std::vector<double> mx::improc::HCIobservation< _realT >::m_RDIimageMJD

Vector of reference image times, in MJD.

Definition at line 257 of file HCIobservation.hpp.

◆ m_RDIkeywords

template<typename _realT >

std::vector<std::string> mx::improc::HCIobservation< _realT >::m_RDIkeywords

Vector of FITS header keywords to read from the files in m_fileList.

Definition at line 396 of file HCIobservation.hpp.

◆ m_RDIqualityFile

template<typename _realT >

std::string mx::improc::HCIobservation< _realT >::m_RDIqualityFile

File containing 2 space-delimited columns of fileMame qualityValue pairs for the reference images.

If this is not empty and m_RDIqualityThreshold is > 0, then only images where qualityValue >= qualityThreshold are read and processed.

The only restriction on m_RDIqualityThreshold is that it is > 0. It is intendend to be something like Strehl ratio.

Definition at line 388 of file HCIobservation.hpp.

◆ m_RDIqualityThreshold

template<typename _realT >

realT mx::improc::HCIobservation< _realT >::m_RDIqualityThreshold { 0 }

Threshold to apply to qualityValues read from qualityFile.

If <= 0, then thresholding is not performed.

Definition at line 393 of file HCIobservation.hpp.

◆ m_refIms

template<typename _realT >

eigenCube<realT> mx::improc::HCIobservation< _realT >::m_refIms

The optional reference image cube.

Definition at line 254 of file HCIobservation.hpp.

◆ m_sigmaThreshold

template<typename _realT >

realT mx::improc::HCIobservation< _realT >::m_sigmaThreshold { 0 }

The standard deviation threshold used if combineMethod == HCI::sigmaMeanCombine.

Definition at line 611 of file HCIobservation.hpp.

◆ m_skipPreProcess

template<typename _realT >

bool mx::improc::HCIobservation< _realT >::m_skipPreProcess { false }

Don't do any of the pre-processing steps (including coadding).

Definition at line 518 of file HCIobservation.hpp.

◆ m_tgtIms

template<typename _realT >

eigenCube<realT> mx::improc::HCIobservation< _realT >::m_tgtIms

The target image cube.

Definition at line 222 of file HCIobservation.hpp.

◆ m_thresholdOnly

template<typename _realT >

bool mx::improc::HCIobservation< _realT >::m_thresholdOnly { false }

Just prints the names and qualities of the files which pass threshold, and stop.

Useful mainly for debugging.

Definition at line 318 of file HCIobservation.hpp.

◆ m_weightFile

template<typename _realT >

std::string mx::improc::HCIobservation< _realT >::m_weightFile

Specifies a file containing the image weights, for combining with weighted mean.

This 2-column space-delimited ASCII file containing filenames and weights. It must be specified before readFiles() is executed. In readFiles this is loaded after the m_fileList is cutdown and matched to the remaining files.

Definition at line 608 of file HCIobservation.hpp.

The documentation for this struct was generated from the following file:

improc/HCIobservation.hpp

Public Types

Public Member Functions

Public Attributes

Target File Reading

Reference File Reading

Coadding

Masking

Pre-Processing

Image Combination

Output

Member Typedef Documentation

◆ eigenImageT

◆ realT

Constructor & Destructor Documentation

◆ HCIobservation() [1/5]

◆ HCIobservation() [2/5]

◆ HCIobservation() [3/5]

◆ HCIobservation() [4/5]

◆ HCIobservation() [5/5]

Member Function Documentation

◆ coaddImages()

◆ combineFinim()

◆ load_fileList() [1/2]

◆ load_fileList() [2/2]

◆ load_RDIfileList() [1/2]

◆ load_RDIfileList() [2/2]

◆ makeMaskCube()

◆ outputPreProcessed()

◆ outputPSFSub()

◆ outputRDIPreProcessed()

◆ postCoadd()

◆ postRDICoadd()

◆ postRDIReadFiles()

◆ postReadFiles()

◆ preProcess()

◆ preProcess_meanSub()

◆ preProcess_pixelTSNorm()

◆ readFiles()

◆ readMask()

◆ readPSFSub()

◆ readRDIFiles()

◆ readWeights()

◆ stdFitsHeader()

◆ threshold()

◆ writeFinim()

Member Data Documentation

◆ m_auxDataDir

◆ m_coaddCombineMethod

◆ m_coaddKeywords

◆ m_coaddMaxImno

◆ m_coaddMaxTime

◆ m_combineMethod

◆ m_comboWeights

◆ m_deleteBack

◆ m_deleteFront

◆ m_doOutputPSFSub

◆ m_doWriteFinim

◆ m_exactFinimName

◆ m_fileList

◆ m_filesDeleted

◆ m_filesRead

◆ m_finim

◆ m_finimName

◆ m_heads

◆ m_imageMJD

◆ m_imSize

◆ m_keywords

◆ m_mask

◆ m_maskCube

◆ m_maskFile

◆ m_minGoodFract

◆ m_MJDisISO8601

◆ m_MJDKeyword

◆ m_MJDUnits

◆ m_moveAuxDataDir

◆ m_Ncols

◆ m_Nims

◆ m_Npix

◆ m_Nrows

◆ m_outputDir