Creating an Alert Broker Module for the TOM Toolkit¶
This guide will walk you through how to create a custom alert broker module using the TOM toolkit.
At the end of this tutorial we will have a very simple module that connects to an “alert broker” (in this case a static json file) and allows us to ingest targets into our TOM.
You can follow this example to build an alert broker module to connect to a real alert broker.
Be sure you’ve followed the Getting Started guide before continuing onto this tutorial.
Read these first!
The following Python/Django concepts are used in this tutorial. While this tutorial does not assume familiarity with the concepts, you will likely find the tutorial easier to understand and build upon if you read these in advance.
TOM Alerts module¶
The TOM Alerts module is a Django app which provides the methods and classes needed to create a custom TOM alert broker module. A module may be created to ingest alerts of an arbitrary form from a remote source. The TOM Alerts module provides tools to transform these alerts into TOM-specific alerts to be used in the creation of TOM Targets.
After following the Getting Started guide, you will have a Django project directory of the form:
mytom ├── db.sqlite3 ├── manage.py └── mytom ├── __init__.py ├── settings.py ├── urls.py └── wsgi.py
Creating a Broker Module¶
In this example, we will create a broker named __MyBroker__.
Begin by creating a file
my_broker.py, and placing it in the inner
of the project (in the directory with settings.py).
my_broker.py will contain the classes that define our custom
TOM Alert Broker Module.
Our custom broker module relies on the TOM Toolkit modules that were installed in the
Getting Started guide. Begin by editing
to import the necessary modules.
from tom_alerts.alerts import GenericQueryForm, GenericAlert, GenericBroker from tom_alerts.models import BrokerQuery from tom_targets.models import Target
In order to add custom forms to our broker module, we will also need Django’s forms module, as well the Python module requests, which will allow us to fetch some remote broker test data.
from django import forms import requests
In place of a remote broker, we’ve uploaded a sample JSON file to GitHub Gist.
my_broker.py module to use this data, we will set
broker_url to it.
broker_url = 'https://gist.githubusercontent.com/mgdaily/f5dfb4047aaeb393bf1996f0823e1064/raw/5e6a6142ff77e7eb783892f1d1d01b13489032cc/example_broker_data.json'
To define the query forms for our custom broker module, we’ll begin by creating class
my_broker.py, which inherits the
This will define the list of forms to be presented within the broker query. For our example, we’ll be querying simply on target name.
class MyBrokerForm(GenericQueryForm): target_name = forms.CharField(required=True)
To define our broker module, we’ll create the class
MyBroker, also inside of
Our broker class will encapsulate the logic for making queries to a remote alert broker,
retrieving and sanitizing data, and creating TOM alerts from it.
Begin by defining the class, its name and default form. In our case, the name
will simply be ‘MyBroker’, and the form will be
MyBrokerForm - the form that we
class MyBroker(GenericBroker): name = 'MyBroker' form = MyBrokerForm
Required Broker Class Methods¶
Each TOM alert broker module is required to have a base set of class methods. These methods enable the conversion of remote alert data into TOM-specific alerts and targets.
fetch_alerts Class Method¶
fetch_alerts is used to query the remote broker, and return both an iterator of results and any broker feedback received depending on the parameters passed into the query, so that any results or feedback (such as error messages) may be displayed on the query results page. In our case, fetch_alerts will only filter on name, but this can be easily extended to other query parameters.
@classmethod def fetch_alerts(clazz, parameters): broker_feedback = '' response = requests.get(broker_url) response.raise_for_status() test_alerts = response.json() alert_list =  try: alert_list = [alert for alert in test_alerts if alert['name'] == parameters['target_name']] except KeyError: # We want to catch error messages returned from the Broker and pass them on as feedback. broker_feedback = test_alerts return iter(alert_list), broker_feedback
Why an iterator? Because some alert brokers work by sending streams, not fully evaluated lists. This simple example broker could easily return a list (in fact we are coercing the list into an iterator!) but that would not work in the model where a broker is sending an unending stream of alerts.
Our implementation will get a response from our test broker source, check that our request was successful, and if so, return a iterator of alerts whose name field matches the name passed into the query. If the keyword ‘name’ isn’t present in the alert, we pass the results as feedback.
to_generic_alert Class Method¶
In order to standardize alerts and display them in a consistent manner,
GenericAlert class has been defined within the
This broker method converts a remote alert into a TOM Toolkit
@classmethod def to_generic_alert(clazz, alert): return GenericAlert( timestamp=alert['timestamp'], url=broker_url, id=alert['id'], name=alert['name'], ra=alert['ra'], dec=alert['dec'], mag=alert['mag'], score=alert['score'] )
In our case, the
GenericAlert attributes match up almost directly with our test
data. How convenient! We’ll just go ahead and define the
field as the
broker_url we retrieved our test data from.
... url=broker_url, ...
to_generic_alert are the only methods required for your
broker module to function. Of course you are free to add any number of additional
methods or attributes to the module that you deem necessary.
Using Our New Alert Broker¶
Now that we’ve created our TOM alert broker, let’s hook it into our TOM so that we can ingest alerts and create targets.
tom_alerts module will look in
settings.py for a list of alert
broker classes, so we’ll need to add
MyBroker to that list.
TOM_ALERT_CLASSES = [ ... 'tom_alerts.brokers.mars.MARSBroker', 'mytom.my_broker.MyBroker', ... ]
Now, navigate to the top-level directory of your Django project,
manage.py resides and run
./manage.py makemigrations ./manage.py migrate ./manage.py runserver
Navigate to http://127.0.0.1:8000/alerts/query/list/
You should now see ‘MyBroker’ listed as a broker! Clicking the link will bring you to the query page, where you can make a query to our sample dataset.
Making a Query¶
Since we’re only going to be filtering on the alert’s ‘target_name’ field, we’re only presented with that option. Name the query whatever you’d like, and we’ll check our remote data source for a target named ‘Tatooine’
Going back to http://127.0.0.1:8000/alerts/query/list/, our new query will appear. Click the ‘run’ button to run the query.
The query result will be presented.
To create a target from any query result, click the ‘create target’ button. To view the raw alert data, click the ‘view’ link.
Click here to view the full source code detailed in this example.