Nifi processing files as zero bytes
Hi,
Iam using Nifi 1.3 version and this is the flow..getsftp->putsftp->putsftp.
But at times after the files are picked by getsftp,say if its 190 bytes then putsftp transfers as 0 bytes.
It doesnot happen all the time but at times.
Please find below the error:
2019-01-07 02:00:01,272 ERROR [Timer-Driven Process Thread-7] o.a.nifi.processors.standard.PutSFTP PutSFTP[id=XXX] PutSFTP[id=XXX] failed to process session due to org.apache.nifi.processor.exception.MissingFlowFileException: Unable to find content for FlowFile: {} org.apache.nifi.processor.exception.MissingFlowFileException: Unable to find content for FlowFile at org.apache.nifi.controller.repository.StandardProcessSession.handleContentNotFound(StandardProcessSession.java:3003) at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2157) at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2106) at org.apache.nifi.processors.standard.PutFileTransfer.onTrigger(PutFileTransfer.java:127) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.nifi.controller.repository.ContentNotFoundException: Could not find content for StandardContentClaim [resourceClaim=StandardResourceClaim[id=XXX, container=default, section=490], offset=0, length=190]: Stream contained only 0 bytes but should have contained 190 at org.apache.nifi.controller.repository.io.FlowFileAccessInputStream.ensureAllContentRead(FlowFileAccessInputStream.java:49) at org.apache.nifi.controller.repository.io.FlowFileAccessInputStream.read(FlowFileAccessInputStream.java:84) at java.io.BufferedInputStream.read1(BufferedInputStream.java:284) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:636) at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:540) at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:492) at org.apache.nifi.processors.standard.util.SFTPTransfer.put(SFTPTransfer.java:543) at org.apache.nifi.processors.standard.PutFileTransfer$1.process(PutFileTransfer.java:135) at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2136) ... 14 common frames omitted 2019-01-07 02:00:01,272 WARN [Timer-Driven Process Thread-7] o.a.nifi.processors.standard.PutSFTP PutSFTP[id=015b10b1-853c-1b95-d98f-915e12c396bd] Processor Administrat ively Yielded for 1 sec due to processing failure 2019-01-07 02:00:01,273 WARN [Timer-Driven Process Thread-7] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding PutSFTP[id=XXX] due to uncaught Exception: org.apache.nifi.processor.exception.MissingFlowFileException: Unable to find content for FlowFile
Can you please help me in finding why the files are transfered as 0 bytes?
I would inspect your content repository to see if the referenced claim (StandardResourceClaim[id=XXX, container=default, section=490], offset=0, length=190]) still exists.
Within content_repository directory, I would look for sub-folder "490". Then within that folder look for file named XXX (assuming you replaced actual claim number here with XXX)
-
It sounds like this file my have been deleted.
Do you have some external process that may be access your content repository?
Was the content repository maybe moved?
Do you maybe have multiple node NiFi cluster where every node is trying to share same mounted content repo?
Was NIFi restarted as different user? This could result in some files in repo being owned by different users which may lead to permissions issues access those files.
-
FlowFiles are what move from processor to processor. The FlowFile metadata (stored in flowfile repository) includes information on the size and location of the physical content within one of the content repositories (default in this case). In this case, the FlowFile has reached a processor where actually retrieving that content was needed, but could not be found.
-
Thank you,
Matt
