Hortonworks.com
  • Explore
    • All Tags
    • All Questions
    • All Articles
    • All Ideas
    • All Repos
    • All SKB
    • All Users
    • All Badges
    • Leaderboard
  • Create
    • Ask a question
    • Create Article
    • Post Idea
    • Add Repo
  • Tracks
    • All Tracks
    • Community Help
    • Cloud & Operations
    • CyberSecurity
    • Data Ingestion & Streaming
    • Data Processing
    • Data Science & Advanced Analytics
    • Design & Architecture
    • Governance & Lifecycle
    • Hadoop Core
    • Sandbox & Learning
    • Security
    • Solutions
  • Login
HCC Hortonworks Community Connection
  • Home /
  • Community Help /
avatar image

Nifi processing files as zero bytes

Question by Gillu Varghese Jan 07 at 12:05 PM nifi-processornifi-streamingnifi-uinifi-repository

Hi,

Iam using Nifi 1.3 version and this is the flow..getsftp->putsftp->putsftp.

But at times after the files are picked by getsftp,say if its 190 bytes then putsftp transfers as 0 bytes.

It doesnot happen all the time but at times.

Please find below the error:

2019-01-07 02:00:01,272 ERROR [Timer-Driven Process Thread-7] o.a.nifi.processors.standard.PutSFTP PutSFTP[id=XXX] PutSFTP[id=XXX] failed to process session due to org.apache.nifi.processor.exception.MissingFlowFileException: Unable to find content for FlowFile: {} org.apache.nifi.processor.exception.MissingFlowFileException: Unable to find content for FlowFile at org.apache.nifi.controller.repository.StandardProcessSession.handleContentNotFound(StandardProcessSession.java:3003) at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2157) at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2106) at org.apache.nifi.processors.standard.PutFileTransfer.onTrigger(PutFileTransfer.java:127) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.nifi.controller.repository.ContentNotFoundException: Could not find content for StandardContentClaim [resourceClaim=StandardResourceClaim[id=XXX, container=default, section=490], offset=0, length=190]: Stream contained only 0 bytes but should have contained 190 at org.apache.nifi.controller.repository.io.FlowFileAccessInputStream.ensureAllContentRead(FlowFileAccessInputStream.java:49) at org.apache.nifi.controller.repository.io.FlowFileAccessInputStream.read(FlowFileAccessInputStream.java:84) at java.io.BufferedInputStream.read1(BufferedInputStream.java:284) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:636) at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:540) at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:492) at org.apache.nifi.processors.standard.util.SFTPTransfer.put(SFTPTransfer.java:543) at org.apache.nifi.processors.standard.PutFileTransfer$1.process(PutFileTransfer.java:135) at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2136) ... 14 common frames omitted 2019-01-07 02:00:01,272 WARN [Timer-Driven Process Thread-7] o.a.nifi.processors.standard.PutSFTP PutSFTP[id=015b10b1-853c-1b95-d98f-915e12c396bd] Processor Administrat ively Yielded for 1 sec due to processing failure 2019-01-07 02:00:01,273 WARN [Timer-Driven Process Thread-7] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding PutSFTP[id=XXX] due to uncaught Exception: org.apache.nifi.processor.exception.MissingFlowFileException: Unable to find content for FlowFile

Can you please help me in finding why the files are transfered as 0 bytes?

Comment

People who voted for this

0 Show 1
10 |6000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 5 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

avatar image Gillu Varghese · Jan 09 at 07:26 AM 0
Share
@Matt Clarke

@Shu Can you please help in resolving it

1 Reply

· Add your reply
  • Sort: 
  • Votes
  • Created
  • Oldest
avatar image

Answer by Matt Clarke · Jan 10 at 03:20 PM

@Gillu Varghese

I would inspect your content repository to see if the referenced claim (StandardResourceClaim[id=XXX, container=default, section=490], offset=0, length=190]) still exists.

Within content_repository directory, I would look for sub-folder "490". Then within that folder look for file named XXX (assuming you replaced actual claim number here with XXX)

-

It sounds like this file my have been deleted.
Do you have some external process that may be access your content repository?
Was the content repository maybe moved?
Do you maybe have multiple node NiFi cluster where every node is trying to share same mounted content repo?
Was NIFi restarted as different user? This could result in some files in repo being owned by different users which may lead to permissions issues access those files.
-
FlowFiles are what move from processor to processor. The FlowFile metadata (stored in flowfile repository) includes information on the size and location of the physical content within one of the content repositories (default in this case). In this case, the FlowFile has reached a processor where actually retrieving that content was needed, but could not be found.

-

Thank you,

Matt

Comment

People who voted for this

0 Show 0 · Share
10 |6000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 5 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

Your answer

Hint: You can notify a user about this post by typing @username

Up to 5 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

82
Followers

Answers Answer & comments

This website uses cookies for analytics, personalisation and advertising. To learn more or change your cookie settings, please read our Cookie Policy. By continuing to browse, you agree to our use of cookies.

HCC Guidelines | HCC FAQs | HCC Privacy Policy | Privacy Policy | Terms of Service

© 2011-2019 Hortonworks Inc. All Rights Reserved.

Hadoop, Falcon, Atlas, Sqoop, Flume, Kafka, Pig, Hive, HBase, Accumulo, Storm, Solr, Spark, Ranger, Knox, Ambari, ZooKeeper, Oozie and the Hadoop elephant logo are trademarks of the Apache Software Foundation.

  • Anonymous
  • Login
  • Create
  • Ask a question
  • Create Article
  • Post Idea
  • Add Repo
  • Create SupportKB
  • Tracks
  • Community Help
  • Cloud & Operations
  • CyberSecurity
  • Data Ingestion & Streaming
  • Data Processing
  • Data Science & Advanced Analytics
  • Design & Architecture
  • Governance & Lifecycle
  • Hadoop Core
  • Sandbox & Learning
  • Security
  • Solutions
  • Explore
  • All Tags
  • All Questions
  • All Articles
  • All Ideas
  • All Repos
  • All SKB
  • All Users
  • Leaderboard
  • All Badges