Introduce generic Callbacks #54796
Introduce generic Callbacks #54796
Conversation
Introduce a generic Callback model that can be used for Deadline callbacks for now and hopefully the on_*_callbacks for Dags and Tasks in the future.
|
I've been working on making Deadline callbacks work on the executor in addition to the Triggerer. That means introducing a new Workload type for the executor. I didn't want to tie it specifically to Deadlines and realized it would be a good idea to introduce generic Callbacks that the new workload can reference. This is an initial PR for adding a new Callback table and refactoring existing Deadline callbacks that run on the Triggerer to use these instead: Ideally, once AIP-92 is in progress, these callbacks can be used or all the |
| @@ -84,6 +83,14 @@ def key(self) -> TaskInstanceKey: | |||
| ) | |||
|
|
|||
|
|
|||
| class Callback(BaseModel): | |||
There was a problem hiding this comment.
Similar to TaskInstance defined above this in this file
| @@ -132,6 +139,51 @@ def make( | |||
| ) | |||
|
|
|||
|
|
|||
| class ExecuteCallback(BaseWorkload): | |||
There was a problem hiding this comment.
Similar to ExecuteTask defined above. There's a lot of duplicate code here that we can refactor but for now, I'm just looking to align on the definition
| EXECUTOR = "executor" | ||
|
|
||
|
|
||
| class Callback(Base): |
There was a problem hiding this comment.
This is the focus of this PR and what I'm looking to align on for all Callbacks
Introduce a generic Callback model that can be used for Deadline callbacks for now and hopefully the on_*_callbacks for Dags and Tasks in the future.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.