https://stackoverflow.com/questions/44403690/empty-output-for-watermarked-aggregation-query-in-append-mode

gistfile1.txt

watermark = 1 hour

First batch (max event time = null):

2017-06-07 10:00:00.000

StateStore will store 2017-06-07 10:00:00.000

Second batch (max event time = 2017-06-07 10:00:00.000):
2017-06-07 11:00:00.000

StateStore will store 2017-06-07 10:00:00.000 and 2017-06-07 11:00:00.000
StateStore will evict rows <= max event time - 1 hour (2017-06-07 09:00:00.000)


Third batch (max event time = 2017-06-07 11:00:00.000):

2017-06-07 12:00:00.000

StateStore will store 2017-06-07 10:00:00.000, 2017-06-07 11:00:00.000 and 2017-06-07 12:00:00.000
StateStore will evict rows <= max event time - 1 hour (2017-06-07 10:00:00.000)

So now you can see 2017-06-07 10:00:00.000 in the output.

Is it possible to shorten the time until data is output by using a negative delayThreshold for the watermark? In my use case, I assign current_timestamp() to eventTime so for me there's no point in waiting for late data to arrive because there is no late data.