Apache Beam

https://beam.apache.org/

Apache Beam provides an advanced unified programming model, allowing you to implement batch and streaming data processing jobs that can run on any execution engine.

Apache Beam is:

  • UNIFIED – Use a single programming model for both batch and streaming use cases.
  • PORTABLE – Execute pipelines on multiple execution environments, including Apache Apex, Apache Flink, Apache Spark, and Google Cloud Dataflow.
  • EXTENSIBLE – Write and share new SDKs, IO connectors, and transformation libraries.