Data Processing¶
# TODO: Fix this
The TOM is configured with built-in data processing on the upload of Data Products. This processing is broken up into three components.
tom_dataproducts.data_processor.run_data_processor
tom_dataproducts.data_processor.DataProcessor
tom_dataproducts.data_serializers
The DataProductUploadView
calls run_data_processor
upon saving the DataProduct
. run_data_processor
instantiates the DATA_PROCESSOR_CLASS
, which can be specified in settings.py
. run_data_processor
then
processes the uploaded DataProduct
based on the tag value, which can either be Photometry or Spectroscopy.
In the case of spectra, the default behavior is that DataProduct
is converted into a specutils.Spectrum1D
from
either a CSV or a FITS file. The spectrum is multiplied by the flux constant of the facility it was taken at, or not
multiplied. The Spectrum1D
is then serialized into JSON for database via the SpectrumSerializer
.
For photometry, the default behavior is simply to read the CSV and convert it to JSON for database storage.
Data Serializers¶
Data Processors¶
-
tom_dataproducts.data_processor.
run_data_processor
(dp)¶ Reads the data_product_type from the dp parameter and imports the corresponding DataProcessor specified in settings.py, then runs process_data and inserts the returned values into the database.
- Parameters
dp (DataProduct) – DataProduct which will be processed into a list
-
class
tom_dataproducts.data_processor.
DataProcessor
¶ -
process_data
(data_product)¶ Routes a photometry processing call to a method specific to a file-format. This method is expected to be implemented by any subclasses.
- Parameters
data_product (DataProduct) – DataProduct which will be processed into a list
- Returns
python list of 2-tuples, each with a timestamp and corresponding data
- Return type
list of 2-tuples
-