Spinning Your Drones wirh Cadence Workflows, Apache KAFKA and Tensorflow - Paul Brebner

Enhance your machine learning models with Cadence workflows, Apache Kafka, and TensorFlow. Learn how to mitigate temporal bias, integrate workflows, and orchestrate complex logic while leveraging infinite streaming data.

Key takeaways

Incremental learning can lead to temporal bias due to fixation on recent samples.
To mitigate this, using rolling or sliding windows to flatten oscillations can be effective.
Cadence workflows are horizontally scalable, supporting unlimited workflows.
Apache Kafka is a choreography system that can be used to integrate workflows.
TensorFlow can be used for machine learning over streaming data.
Streaming data is infinite, so access to all past data is not possible.
State change history is written to a database to enable workflow state recovery in case of failure.
Cadence workflows support slow and long-running flows, sleep and scheduled events, and complex flow logic.
Remote koals can fail, so activities are used to wrap them.
Cadence is slow and can be overwhelmed by complex workflows.
Orchestrating workflows is like synchronously conducting performers to perform in time.
Machine learning models can be evaluated using a confusion matrix.
Using incremental learning can lead to better results than batch learning.
Re-training using new data can help remove pre-drift model inertia.
The accuracy of machine learning models can drop when the model is not updated.
Using Apache Kafka with TensorFlow can enable machine learning over streaming data.
Cadence workflows can be used to integrate Kafka and TensorFlow.
The accuracy of machine learning models can be improved by using rolling or sliding windows to flatten oscillations.

Spinning Your Drones wirh Cadence Workflows, Apache KAFKA and Tensorflow - Paul Brebner

More talks