Hortonworks.com
  • Explore
    • All Tags
    • All Questions
    • All Repos
    • All SKB
    • All Articles
    • All Ideas
    • All Users
    • All Badges
    • Leaderboard
  • Create
    • Ask a question
    • Add Repo
    • Create Article
    • Post Idea
  • Tracks
    • All Tracks
    • Community Help
    • Cloud & Operations
    • CyberSecurity
    • Data Ingestion & Streaming
    • Data Processing
    • Data Science & Advanced Analytics
    • Design & Architecture
    • Governance & Lifecycle
    • Hadoop Core
    • Sandbox & Learning
    • Security
    • Solutions
  • Login
HCC Hortonworks Community Connection
  • Home /
  • Cloud & Operations /
  • Home /
  • Cloud & Operations /
avatar image

Swappiness setting recommendation

  • Export to PDF
emaxwell created · May 17, 2016 at 01:05 PM · edited · May 16, 2016 at 06:04 PM
7

Short Description:

Update to swappiness recommendation on Linux systems

Article

Virtual Memory swapping can have a large impact on the performance of a Hadoop system. Because of the memory requirements of YARN containers and processes running on the nodes in a cluster, swapping process out of memory to disk can cause serious performance limitations. As such, the historical recommendations for setting the swappiness, or propensity to swap out a process, on a Hadoop system has been to disable swap altogether. With newer versions of the Linux kernel, Out Of Memory (OOM) situations can be more likely to indiscriminately kill important processes to reclaim valuable physical memory on the system with a swappiness of 0.

In order to prevent the system from swapping processes too frequently, but still allow for emergency swapping (instead of killing processes), the recommendation is now to set swappiness to 1 on Linux systems. This will still allow swapping, but with the least possible aggressiveness (for comparison, the default value for swappiness is 60).

To change the swappiness on a running machine, use the following command:

echo "1" > /proc/sys/vm/swappiness

To ensure the swappiness is set appropriately on reboot, use the following command:

echo "vm.swappiness=1" >> /etc/sysctl.conf
thub.nodes.view.add-new-comment
faqfaqfaqhelpmemoryoperationsoptimization
Add comment
10 |6000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Article

Contributors

avatar image

Follow

Follow

avatar image avatar image avatar image avatar image avatar image

Navigation

Swappiness setting recommendation

Related Articles

Kafka Topics and Brokers Metrics with Ambari 2.2.2

Optimizing HBase I/O for Large Scale Hadoop Implementations

Introducing Cloudbreak 2.9.0 GA

Google Cloud Platform fix for Cloudbreak - upgrade now to 2.4.1 or 1.16.6

Cloudbreak Credential Troubleshooting Tips

Get Started with HDP 2.6 in HDCloud for AWS 1.14.1

Design and Deployment Considerations for High Availability Kafka Service

How to remove all previous version HDP directories from /usr/hdp using Ambari.

All about the HDP-GPL repository

Recommendations for Microsoft Azure HDP Deployment

This website uses cookies for analytics, personalisation and advertising. To learn more or change your cookie settings, please read our Cookie Policy. By continuing to browse, you agree to our use of cookies.

HCC Guidelines | HCC FAQs | HCC Privacy Policy | Privacy Policy | Terms of Service

© 2011-2019 Hortonworks Inc. All Rights Reserved.

Hadoop, Falcon, Atlas, Sqoop, Flume, Kafka, Pig, Hive, HBase, Accumulo, Storm, Solr, Spark, Ranger, Knox, Ambari, ZooKeeper, Oozie and the Hadoop elephant logo are trademarks of the Apache Software Foundation.

  • Anonymous
  • Login
  • Create
  • Ask a question
  • Add Repo
  • Create SupportKB
  • Create Article
  • Post Idea
  • Tracks
  • Community Help
  • Cloud & Operations
  • CyberSecurity
  • Data Ingestion & Streaming
  • Data Processing
  • Data Science & Advanced Analytics
  • Design & Architecture
  • Governance & Lifecycle
  • Hadoop Core
  • Sandbox & Learning
  • Security
  • Solutions
  • Explore
  • All Tags
  • All Questions
  • All Repos
  • All SKB
  • All Articles
  • All Ideas
  • All Users
  • Leaderboard
  • All Badges