Spark Streaming brings Apache Spark’s language-integrated API to stream processing, letting you write streaming jobs the same way you write batch jobs. It supports Java, Scala and Python. Spark Streaming recovers both lost work and operator state (e.g. sliding windows) out of the box, without any extra code on your part.
Customer Reviews
Administrator in Computer Software
Advanced user of Spark StreamingSpark docs are saying, it is not atomic but it is near-atomic in the way of committing data but it loses in consistency. In our case, we are using a spark engine to read JSON from Apache kafka and write it to Data lake properties. Best part is streaming application on batch based.
It is not good in processing in-memory data when it auto-commit is disabled.
We are doing event processing and analysis using Spark by reading leads from Kafka to streaming to Datalake ecosystem to segregate value data for analytics
