Store and compare timestamp in Nifi
I have a processor group where I read page ID's from a file and retrieve all posts and comments for these pages, I run this every 2 hours. In the response body I get the timestamp of the created time of the post, I need to record the last largest timestamp of all the posts I received and pull only posts .
One approach I though of was to record the timestamp attribute of each flowfile in a file and compare the value of this file with the next flowfiles timestamp, will this work, if not how could I achieve this.
You can use the state and conditions features of the UpdateAttribute processor to implement this
Thanks, I have the timestamp value in each flow file, I need to compare the timestamp of the current flow files with previous flow files and if it is greater write that to a file using the Put file processor. I tried this with the update attribute processor's advanced features in the following way:
Added a rule named max_timestamp with the condition ${getStateValue("maxTimestamp"):lt(${timestamp})} and set the action for the attribute maxTimestamp as ${timestamp}
I also turned set the store state feature to store locally in the processor, but I do not see the maxTimestamp attribute being set to the flowfiles output from the update attribute processor.
In your screenshot you are using 'equals' as a condition rather than 'lt'. Make sure to use the right relation.
Also, you need to add the attribute maxTimestamp to the flowfile.
