Bucket Access Management

It is not recommended to use S3 access control list (ACL) anymore. Managing S3 bucket and object ACLs is cumbersome, limited, and does not scale.

Recommendation: Use S3 bucket policies whenever possible.

Bucket Policy

S3 bucket policy allows you to grant access to your bucket to other projects. Access policies are more fine-grained, more scalable, and allow a much better control of your data.

References

Limitations

  • It is only possible to grant access right at the project level.
  • It is NOT possible to grant access right directly to a user, but only to the project the user is member of.
  • Only S3 bucket policy is available, S3 user policy is not implemented.
  • Identify projects by:
    • Ceph-based: project ID, not project name
    • Cloudian-based: Canonical ID

In a S3 access policy file, the project accessing your bucket must be identified using a Principal identifier. See AWS Principals

Get Project Identifiers

The Cloudian Hyperstore S3 backend uses the CanonicalUser as the principle identifier. You can get the ID of your project by:

  • Log in to SWITCHengines
  • Project -> Object Storage (S3) -> Dashboard
  • View profile and get the Canonical ID from the user overview
    • Select: User Icon -> Profile

The Ceph Object Gateway is natively included within our Openstack infrastructure. You can get the Openstack project ID (which is the principal identifier for S3) for this backend by:

  • Openstack CLI
    openstack project show -c id -f value <your project name>
  • SWITCHengines dashboard:
    • Identity -> Projects : "Project ID"


Depending on your S3 backend, you have to use as Principal:

  • Openstack project ID  (Ceph-based, Region: ZH/LS)
    "Principal": {
    "AWS": "arn:aws:iam::OPENSTACK_PROJECT_ID:root"
    }
  • Cloudian Canonical ID (Cloudian-based, Region: ZH)
    "Principal": {
    "CanonicalUser": "CLOUDIAN_CANONICAL_ID"
    }

as the principle identifier. 

In the following examples, you have to use the principal identifier to refer to your projects.

Exemplary Bucket Policy Configurations

Policy: Anonymous Read Only Bucket and Objects

If the project owner (e.g. OWNER_PROJECT) of the bucket (e.g. SHARED_BUCKET) want to share it read only with every projects/users, the following policy can be used:

{
"Version": "2012-10-17",
"Id": "read-only",
"Statement": [
{
"Sid": "project-read",
"Effect": "Allow",
"Principal": {
"CanonicalUser": "*"
},
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::SHARED_BUCKET",
"arn:aws:s3:::SHARED_BUCKET/*"
]
}
]
}
  • the element "Version": "2012-10-17" is mandatory.
{
"Version": "2012-10-17",
"Id": "read-only",
"Statement": [
{
"Sid": "project-read",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::*"
]
}
]
}
  • the element "Version": "2012-10-17" is mandatory.

Then the bucket's owner uses the s3cmd command to apply and check the policy on the bucket:

s3cmd -c s3cfg-OWNER_PROJECT setpolicy read-only.json s3://SHARED_BUCKET
s3cmd -c s3cfg-OWNER_PROJECT info s3://SHARED_BUCKET
Policy: Read Only Bucket and Objects

If the project owner (e.g. OWNER_PROJECT) of the bucket (e.g. SHARED_BUCKET) want to share it read only with another project (e.g. READ_ONLY_PROJECT), the following policy can be used:

{
"Version": "2012-10-17",
"Id": "read-only",
"Statement": [
{
"Sid": "project-read",
"Effect": "Allow",
"Principal": {
"CanonicalUser": "READ_ONLY_PROJECT_ID"
},
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::SHARED_BUCKET",
"arn:aws:s3:::SHARED_BUCKET/*"
]
}
]
}
  • READ_ONLY_PROJECT_ID is the principle identifier of the project with read only access to the bucket.
  • the element "Version": "2012-10-17" is mandatory. 
{
"Version": "2012-10-17",
"Id": "read-only",
"Statement": [
{
"Sid": "project-read",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::READ_ONLY_PROJECT_ID:root"
},
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::*"
]
}
]
}
  • READ_ONLY_PROJECT_ID is the principle identifier of the project with read only access to the bucket.
  • the element "Version": "2012-10-17" is mandatory. 

Then the bucket's owner uses the s3cmd command to apply and check the policy on the bucket:

s3cmd -c s3cfg-OWNER_PROJECT setpolicy read-only.json s3://SHARED_BUCKET
s3cmd -c s3cfg-OWNER_PROJECT info s3://SHARED_BUCKET
Policy: Read Only Bucket Restricted to "sub-directory" Objects

If the project owner (e.g. OWNER_PROJECT) of the bucket (e.g. SHARED_BUCKET) want to share read only a sub-directory structure with another project (e.g. READ_ONLY_PROJECT), the following policy can be used:

{
  "Version": "2012-10-17",
  "Id": "read-only-subdirectory",
  "Statement": [
    {
      "Sid": "project-list_bucket",
      "Effect": "Allow",
      "Principal": {
       "CanonicalUser": "READ_ONLY_PROJECT_ID"
      },
      "Action": [
       "s3:ListBucket"
      ],
      "Resource": [
"arn:aws:s3:::SHARED_BUCKET",
       "arn:aws:s3:::SHARED_BUCKET/*"
      ]
    },
    {
     "Sid": "project-read_subdirectory",
     "Effect": "Allow",
     "Principal": {
      "CanonicalUser": "READ_ONLY_PROJECT_ID"
},
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::SHARED_BUCKET/foo/bar/*"
]
}
]
}
  • the project-list_bucket statement is not mandatory, but without it the read only project can not list the bucket's objects, and therefore have to know the object path in advance.
  • READ_ONLY_PROJECT_ID is the principle identifier of the project with read only access to the bucket.
  • SHARED_BUCKET/foo/bar/* is the bucket/objects sub-directory prefix to restrict access to.
  • the element "Version": "2012-10-17" is mandatory.
{
  "Version": "2012-10-17",
  "Id": "read-only-subdirectory",
  "Statement": [
    {
      "Sid": "project-list_bucket",
      "Effect": "Allow",
      "Principal": {
       "AWS": "arn:aws:iam::READ_ONLY_PROJECT_ID:root"
      },
      "Action": [
       "s3:ListBucket"
      ],
      "Resource": [
       "arn:aws:s3:::*"
      ]
    },
    {
     "Sid": "project-read_subdirectory",
     "Effect": "Allow",
     "Principal": {
      "AWS": "arn:aws:iam::READ_ONLY_PROJECT_ID:root"
},
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::SHARED_BUCKET/foo/bar/*"
]
}
]
}
  • the project-list_bucket statement is not mandatory, but without it the read only project can not list the bucket's objects, and therefore have to know the object path in advance.
  • READ_ONLY_PROJECT_ID is the principle identifier of the project with read only access to the bucket.
  • SHARED_BUCKET/foo/bar/* is the bucket/objects sub-directory prefix to restrict access to.
  • the element "Version": "2012-10-17" is mandatory. 

Then the bucket's owner uses the s3cmd command to apply and check the policy on the bucket:

s3cmd -c s3cfg-OWNER_PROJECT setpolicy read-only-subdirectory.json s3://SHARED_BUCKET
s3cmd -c s3cfg-OWNER_PROJECT info s3://SHARED_BUCKET
Policy: Read and Write Bucket and Objects

If the project owner (e.g. OWNER_PROJECT) of the bucket (e.g. SHARED_BUCKET) want to share it read and write with another project (e.g. READ_ONLY_PROJECT), the following policy can be used:

{
"Version": "2012-10-17",
"Id": "read-write",
"Statement": [
{
"Sid": "project-read_write",
"Effect": "Allow",
"Principal": {
"CanonicalUser": [
"OWNER_PROJECT_ID",
"READ_WRITE_PROJECT_ID"
]
},
"Action": [
"s3:ListBucket",
"s3:PutObject",
"s3:DeleteObject",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::SHARED_BUCKET",
"arn:aws:s3:::SHARED_BUCKET/*"
]
}
]
}
  • OWNER_PROJECT_ID is the principle identifier of the project sharing the bucket. The bucket owner must be listed explicitly in the Principal AWS list.
  • READ_WRITE_PROJECT_ID is the principle identifier of the project with read and write access to the bucket.
  • the element "Version": "2012-10-17" is mandatory. 
{
"Version": "2012-10-17",
"Id": "read-write",
"Statement": [
{
"Sid": "project-read_write",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::OWNER_PROJECT_ID:root",
"arn:aws:iam::READ_WRITE_PROJECT_ID:root"
]
},
"Action": [
"s3:ListBucket",
"s3:PutObject",
"s3:DeleteObject",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::*"
]
}
]
}
  • OWNER_PROJECT_ID is the principle identifier of the project sharing the bucket. The bucket owner must be listed explicitly in the Principal AWS list.
  • READ_WRITE_PROJECT_ID is the principle identifier of the project with read and write access to the bucket.
  • the element "Version": "2012-10-17" is mandatory. 

Then the bucket's owner uses the s3cmd command to apply and check the policy on the bucket:

s3cmd -c s3cfg-OWNER_PROJECT setpolicy read-write.json s3://SHARED_BUCKET
s3cmd -c s3cfg-OWNER_PROJECT info s3://SHARED_BUCKET