Hortonworks.com
  • Explore
    • All Tags
    • All Questions
    • All Repos
    • All SKB
    • All Articles
    • All Ideas
    • All Repos
    • All SKB
    • All Articles
    • All Ideas
    • All Users
    • All Badges
    • Leaderboard
  • Create
    • Ask a question
    • Add Repo
    • Create Article
    • Post Idea
    • Add Repo
    • Create Article
    • Post Idea
  • Tracks
    • All Tracks
    • Community Help
    • Cloud & Operations
    • CyberSecurity
    • Data Ingestion & Streaming
    • Data Processing
    • Data Science & Advanced Analytics
    • Design & Architecture
    • Governance & Lifecycle
    • Hadoop Core
    • Sandbox & Learning
    • Security
    • Solutions
  • Login
HCC Hortonworks Community Connection
  • Home /
  • Hadoop Core /
avatar image

ACL On Group Level

Question by Mudassar Hussain Jan 12 at 11:19 AM hadoopadministrationuser-groups

Hi Guys,
I have a group "Marketing" and it has 3 users
1. Mark1
2. Mark2
3. Mark3
and other group is "Account" and it has also 3 users
1. AC1
2. AC2
3. AC3
my questions are :
1. How i get the list of all user in any group ? I have tried below command it give me all users and all groups but not specifically.
"cat /etc/passwd | awk -F':' '{ print $1}' | xargs -n1 groups" .
2. I want to set the ACL in such way, In first scenario, User "AC1" (group: "Account") will have the Right "Read/Write/Execute" in Group "Marketing".
3. In second scenario, User Mark1 not able to copy the file into "Account" user.
Please guide me in details.
NOTE : I have been using Amazon Machine.
Thanks,

Comment

People who voted for this

0 Show 0
10 |6000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

8 Replies

· Add your reply
  • Sort: 
  • Votes
  • Created
  • Oldest
avatar image
Best Answer

Answer by Geoffrey Shelton Okot · Jan 12 at 09:57 PM

@Mudassar Hussain

Prerequisite for question 1,2 and 3

I am assuming you are creating the ACL's from scratch, below are steps to prepare the groups and users

Create the 2 groups

# groupadd Marketing
# groupadd Account 

Add the 3 users to Marketing group

# useradd -G  Marketing Mark1 
# useradd -G  Marketing Mark2
# useradd -G  Marketing Mark3 

Add 3 users to Accounting group

# useradd -G Account AC1
# useradd -G Account AC2
# useradd -G Account AC3 

Answer to question 1

There are 2 variations to get the all memebers of a group in linux the 2 versions of the command are below
grep 'Account' /etc/group
awk -F':' '/Marketing/{print $4}' /etc/group 

Expected output

[root@nakuru ~]# grep 'Account' /etc/group
Account:x:1029:AC1,AC2,AC3
[root@nakuru ~]# awk -F':' '/Marketing/{print $4}' /etc/group
Mark1,Mark2,Mark3 

To enable ACL's in HDP you need to set the dfs.namenode.acls.enabled to true using Ambari in custom hdfs-site.xml which is the recommended way. And restart all stale service typicall HDFS,MapReduce,YARN,ATLAS in my case see attached screenshot

Answer to question 2

Task Set user "AC1" (group: "Account") to have "Read/Write/Execute" privilege in Group "Marketing".

This will entail creating a file in hdfs with owner Mark1or 2 or 3 and group Marketing, as root switch to any user in group Marketing. First create a directory in hdfs and change the ownership to Mark1 and group Marketing

As hdfs user created the directory and change ownership and permission

# su - hdfs [hdfs@nakuru ~]
$ hdfs dfs -mkdir -p /marketing/acldemo 
[hdfs@nakuru ~]$ hdfs dfs -chown -R Mark1:marketing /marketing/acldemo 

Validate the above commands were successful.

[hdfs@nakuru ~]$ hdfs dfs -ls /marketing 
Found 1 items drwxr-xr-x - Mark1 marketing 0 2018-01-12 21:54 /marketing/acldemo 

Get the current ACL

[hdfs@nakuru ~]$ hdfs dfs -getfacl -R /marketing/acldemo 
# file: /marketing/acldemo 
# owner: Mark1 
# group: marketing 
user::rwx 
group::r-x 
other::r-x 

I removed the r-x for other to be sure and revalidate note the others now had no r-x

[Mark1@nakuru ~]$ hdfs dfs -chmod 750 /marketing/acldemo 
[Mark1@nakuru ~]$ hdfs dfs -getfacl -R /marketing/acldemo 
# file: /marketing/acldemo 
# owner: Mark1 
# group: marketing 
user::rwx 
group::r-x 
other::--- 

Switch to user Mark1 create a local file and copy to

hdfs # su - Mark1 
[Mark1@nakuru ~]$ echo "This is Hussain testing ACL ser "AC1" (group: "Account") will have the Right Read/Write/Execute in Group "Marketing"" > test1.txt 
[Mark1@nakuru ~]$ ls -al -rw-r--r-- 1 Mark1 Marketing 113 Jan 12 21:51 test1.txt 

Copy the above file to hdfs in previously created directory and check that it was successfully copied to hdfs

[Mark1@nakuru ~]$ hdfs dfs -put test1.txt /marketing/acldemo 
[Mark1@nakuru ~]$ hdfs dfs -ls /marketing/acldemo 
Found 1 items -rw-r--r-- 3 Mark1 marketing 113 2018-01-12 22:05 /marketing/acldemo/test1.txt 

Testing

Switched to user AC1 in group Account to see if he could read the file, it failed that's normal

[root@nakuru ~]# su AC1 
[AC1@nakuru root]$ hdfs dfs -cat /marketing/acldemo/test1.txt 
cat: Permission denied: user=AC1, access=EXECUTE, inode="/marketing/acldemo/test1.txt":Mark1:marketing:drwxr-x- 

Change the ACL for user AC1 of group Account to have rwx as you requested

[Mark1@nakuru ~]$ hdfs dfs -setfacl -m user:AC1:rwx /marketing/acldemo 

Check the new ACL,note now the user ACI now has rwx on the file test1

[Mark1@nakuru ~]$ hdfs dfs -getfacl /marketing/acldemo/test1.txt 
# file: /marketing/acldemo/test1.txt 
# owner: Mark1 
# group: marketing 
user::rw- 
user:AC1:rwx 
group::r-- 
mask::rwx 
other::r-- 

Switch to user AC1 and test that user AC1 can now read the file.

[root@nakuru ~]# su AC1 
[AC1@nakuru root]$ hdfs dfs -cat /marketing/acldemo/test1.txt 
This is Hussain testing ACL ser AC1 (group: Account) will have the Right Read/Write/Execute in Group Marketing 

SUCCESS !

Answer to question 3

User Mark1 of Marketing should not able to copy the file into "Account" user, create directory and change ownership to any user in Account group

[root@nakuru ~]# su - hdfs 
[hdfs@nakuru ~]$ hdfs dfs -mkdir -p /Account/acldemo2 
[hdfs@nakuru ~]$ hdfs dfs -chown AC1:Account /Account/acldemo2 

Get the ACL of newly created directory, note the 3 octets (other is r-x)

[root@nakuru ~]# su AC1 
[AC1@nakuru root]$ hdfs dfs -getfacl /Account/acldemo2 
# file: /Account/acldemo2 
# owner: AC1 
# group: Account 
user::rwx 
group::r-x 
other::r-x 

Test with user Mark1 can't copy a file to the directory /Account/acldemo2 from local

[root@nakuru ~]# su - Mark1 
[Mark1@nakuru ~]$ hdfs dfs -put test1.txt /Account/acldemo2 
put: Permission denied: user=Mark1, access=WRITE, inode="/Account/acldemo2/test1.txt._COPYING_":AC1:Account:drwxr-xr-x 

The above is quite straightforward Mark1 belongs to Marketing and doesn't have any permissions on this directory, I hope that's what you meant?

Please if that answers your question then please Accept the answer by Clicking on Accept button below, That would be a great help to Community users to find a solution quickly for these kinds of ACL issues.

Comment

People who voted for this

0 Show 0 · Share
10 |6000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image

Answer by Geoffrey Shelton Okot · Jan 14 at 10:56 PM

@Mudassar Hussain

Did it resolve your sitaution?

Comment

People who voted for this

0 Show 0 · Share
10 |6000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image

Answer by Mudassar Hussain · Jan 16 at 11:19 AM

Thanks a lot @Geoffrey Shelton Okot for your brief answer. Sorry for late reply!
I am unable to see the All user of the Group. please see the attache image :


group.jpg (30.3 kB)
Comment

People who voted for this

0 Show 0 · Share
10 |6000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image

Answer by Mudassar Hussain · Jan 16 at 11:57 AM

@Geoffrey Shelton Okot

I have been using amazon machine. node detail :
1. ResourceManager
2. HiveServer
3. ResourceManager
4. Node2
5. Node1 ( I have added this node into cluster)
Thanks

Comment

People who voted for this

0 Show 0 · Share
10 |6000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image

Answer by Mudassar Hussain · Jan 16 at 01:30 PM

Currently "Resource Manager High Availability" just on ResourceManager.I can add AdditionalResourceManager on "NameNode"

Everything set already on AWS machine. I am using it for exam learning.

Comment

People who voted for this

0 Show 0 · Share
10 |6000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image

Answer by Geoffrey Shelton Okot · Jan 16 at 11:53 AM

@Mudassar Hussain

I see you are failing on the namenode. Whats your cluster setup (node distribution) single or multinode cluster ? Typical your user should have been created on the gateway node.


Please revert

Comment

People who voted for this

0 Show 0 · Share
10 |6000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image

Answer by Geoffrey Shelton Okot · Jan 16 at 01:01 PM

@Mudassar Hussain

Do you plan to have Resource Manager HA only and not a Namenode HA? How many physical servers in AWS do you plan to deploy? Here is a typical setup looks like this

1.Gateway aka edge node

2.Master nodes best is more than one for NN HA and RM HA etc

3.Slave node aka data nodes (As many as possible)

Are you using a blueprint to deploy in AWS?

Comment

People who voted for this

0 Show 0 · Share
10 |6000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image

Answer by Geoffrey Shelton Okot · Jan 16 at 01:34 PM

@Mudassar Hussain

Ok good go ahead and do the setup and most probably we could do a remote session to check the ACL stuff.

Please keep me posted.

Comment

People who voted for this

0 Show 0 · Share
10 |6000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Your answer

Hint: You can notify a user about this post by typing @username

Up to 5 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

63
Followers
follow question

Answers Answer & comments

HCC Guidelines | HCC FAQs | HCC Privacy Policy

Hortonworks - Develops, Distributes and Supports Open Enterprise Hadoop.

© 2011-2017 Hortonworks Inc. All Rights Reserved.
Hadoop, Falcon, Atlas, Sqoop, Flume, Kafka, Pig, Hive, HBase, Accumulo, Storm, Solr, Spark, Ranger, Knox, Ambari, ZooKeeper, Oozie and the Hadoop elephant logo are trademarks of the Apache Software Foundation.
Privacy Policy | Terms of Service

HCC Guidelines | HCC FAQs | HCC Privacy Policy | Privacy Policy | Terms of Service

© 2011-2018 Hortonworks Inc. All Rights Reserved.

Hadoop, Falcon, Atlas, Sqoop, Flume, Kafka, Pig, Hive, HBase, Accumulo, Storm, Solr, Spark, Ranger, Knox, Ambari, ZooKeeper, Oozie and the Hadoop elephant logo are trademarks of the Apache Software Foundation.

  • Anonymous
  • Login
  • Create
  • Ask a question
  • Add Repo
  • Create SupportKB
  • Create Article
  • Post Idea
  • Add Repo
  • Create SupportKB
  • Create Article
  • Post Idea
  • Tracks
  • Community Help
  • Cloud & Operations
  • CyberSecurity
  • Data Ingestion & Streaming
  • Data Processing
  • Data Science & Advanced Analytics
  • Design & Architecture
  • Governance & Lifecycle
  • Hadoop Core
  • Sandbox & Learning
  • Security
  • Solutions
  • Explore
  • All Tags
  • All Questions
  • All Repos
  • All SKB
  • All Articles
  • All Ideas
  • All Repos
  • All SKB
  • All Articles
  • All Ideas
  • All Users
  • Leaderboard
  • All Badges