Cloud Dataflow is a fully-managed service for transforming and enriching data in stream (real time) and batch (historical) modes with equal reliability and expressiveness — no more complex workarounds or compromises needed. And with its serverless approach to resource provisioning and management, you have access to virtually limitless capacity to solve your biggest data processing challenges, while paying only for what you use.
Customer Reviews
Consultant in Airlines/Aviation
Advanced user of Google Cloud DataflowThis tool based on well known Apache Beam framework and can be locally or on internal GCP runner in GCP cloud. Has a great set of instruments to work with Stream data and Batch data in very fast manner.
It is very easy to run pipeline and track progress of the job. Much more powerful and easy than Dataproc with native Spark.
Not so clear documentation. You should practice a lot with that to pick up configuration for your particular case.
You should definitely consider this tool in cases of BigData processing. You will have single tool for both Stream and Batch data pipelines
Processing huge number of JSON data both as Stream and Batch data pipelines
