What defines a batch pipeline?

A batch pipeline is primarily characterized by its ability to process and analyze large sets of data at once, rather than dealing with data in a continuous, real-time manner. This often involves scheduled runs where the pipeline will process the entire dataset or a significant portion of it.

The reason that the option indicating it recomputes every dataset that has changed on each run is correct lies in the nature of how batch processes are typically designed. In a batch pipeline, each execution will assess and potentially reprocess any new or modified data since the last execution. This is crucial for ensuring that the results are up to date and accurately reflect any changes that may have occurred over time.

Continuous and real-time processing is more aligned with stream processing systems, which focus on data as it comes in, rather than waiting for a predetermined time to process batches of data.

Batch processing also does not only focus on new data; it typically encompasses the entire dataset relevant to the operation being conducted. While minimizing compute costs can be a side benefit of careful batch processing, it is not a defining characteristic of what makes a batch pipeline. The central aspect remains the complete recomputation based on all changes that have occurred since the last run, ensuring data integrity and completeness in processing.