Apache Beam

What do you like best?

Liked the way Beam provides abstraction for the complex distributed paradigm using pCollection and Transforms etc.

Also its unified approach to deal with both Batch and Stream processing is unique and efficient.

What do you dislike?

At the time of doing development , i found doing Join operation was not that simple and we had to use coGroupByKey for that which was a bit confusing for us. May be providing abstraction as simple join operation would help the users community. May be the developer community might have already provided that option but it was not there when we were doing our use case.

What problems are you solving with the product? What benefits have you realized?

We have been using Beam for a batch data processing with Source as Google Cloud Storage and Destination is Big Query.

We found it very efficient as we did various tranformations on the fly as well.