Bucket Access Management

It is not recommended to use S3 access control list (ACL) anymore. Managing S3 bucket and object ACLs is cumbersome, limited, and does not scale.

Recommendation: Use S3 bucket policies whenever possible.

Bucket Policy

S3 bucket policy allows you to grant access to your bucket to other projects. Access policies are more fine-grained, more scalable, and allow a much better control of your data.

References

Limitations

  • It is only possible to grant access right at the project level.
  • It is NOT possible to grant access right directly to a user, but only to the project the user is member of.
  • Only S3 bucket policy is available, S3 user policy is not implemented.
  • Identify projects by:
    • Ceph-based: project ID, not project name
    • Cloudian-based: Canonical ID

In a S3 access policy file, the project accessing your bucket must be identified using a Principal identifier. See AWS Principals

Get Project Identifiers

The Cloudian Hyperstore S3 backend uses the CanonicalUser as the Principal identifier. You can get the Cloudian Canonical ID of your project by:

  • Log in to SWITCHengines
  • Project -> Object Storage (S3) -> Dashboard
  • View profile and get the Canonical ID from the user overview
    • Select: User Icon -> Profile

The Ceph Object Gateway is natively included within our Openstack infrastructure. You can get the Openstack project ID (which is the principal identifier for S3) for this backend by:

  • Openstack CLI
    openstack project show -c id -f value <your project name>
  • SWITCHengines dashboard:
    • Identity -> Projects : "Project ID"


Depending on your S3 backend (Cloudian S3, or Ceph S3), you have to use as Principal:

  • Cloudian Canonical ID (Cloudian S3 - Object Storage, Region: ZH)
    "Principal": {
    "CanonicalUser": "CLOUDIAN_CANONICAL_ID"
    }
  • Openstack project ID  (Ceph S3, Region: ZH/LS)
    "Principal": {
    "AWS": "arn:aws:iam::OPENSTACK_PROJECT_ID:root"
    }

as the principle identifier. The Cloudian Canonical ID is NOT the same as the OpenStack project ID.

In the following examples, you have to use the principal identifier to refer to your projects.

Exemplary Bucket Policy Configurations

Policy: Anonymous Read Only Bucket and Objects

If the project owner (e.g. OWNER_PROJECT) of the bucket (e.g. SHARED_BUCKET) want to share it read only with every projects/users, the following policy can be used:

{
"Version": "2012-10-17",
"Id": "read-only",
"Statement": [
{
"Sid": "project-read",
"Effect": "Allow",
"Principal": {
"CanonicalUser": "*"
},
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::SHARED_BUCKET",
"arn:aws:s3:::SHARED_BUCKET/*"
]
}
]
}
  • the element "Version": "2012-10-17" is mandatory.
{
"Version": "2012-10-17",
"Id": "read-only",
"Statement": [
{
"Sid": "project-read",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::*"
]
}
]
}
  • the element "Version": "2012-10-17" is mandatory.

Then the bucket's owner uses the s3cmd command to apply and check the policy on the bucket:

s3cmd -c s3cfg-OWNER_PROJECT setpolicy read-only.json s3://SHARED_BUCKET
s3cmd -c s3cfg-OWNER_PROJECT info s3://SHARED_BUCKET
Policy: Read Only Bucket and Objects

If the project owner of the bucket (e.g. SHARED_BUCKET) want to share it read only with another project (e.g. READ_ONLY_CLOUDIAN_CANONICAL_ID, READ_ONLY_PROJECT_ID), the following policy can be used:

{
"Version": "2012-10-17",
"Id": "read-only",
"Statement": [
{
"Sid": "project-read",
"Effect": "Allow",
"Principal": {
"CanonicalUser": "READ_ONLY_CLOUDIAN_CANONICAL_ID"
},
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::SHARED_BUCKET",
"arn:aws:s3:::SHARED_BUCKET/*"
]
}
]
}
  • READ_ONLY_CLOUDIAN_CANONICAL_ID is the principle identifier of the project with read only access to the bucket.
  • the element "Version": "2012-10-17" is mandatory. 
{
"Version": "2012-10-17",
"Id": "read-only",
"Statement": [
{
"Sid": "project-read",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::READ_ONLY_PROJECT_ID:root"
},
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::*"
]
}
]
}
  • READ_ONLY_PROJECT_ID is the principle identifier of the project with read only access to the bucket.
  • the element "Version": "2012-10-17" is mandatory. 

Then the bucket's owner uses the s3cmd command to apply and check the policy on the bucket:

s3cmd -c s3cfg-OWNER_PROJECT setpolicy read-only.json s3://SHARED_BUCKET
s3cmd -c s3cfg-OWNER_PROJECT info s3://SHARED_BUCKET
Policy: Read Only Bucket Restricted to "sub-directory" Objects

If the project owner of the bucket (e.g. SHARED_BUCKET) want to share read only a sub-directory structure with another project (e.g. READ_ONLY_CLOUDIAN_CANONICAL_ID, READ_ONLY_PROJECT_ID), the following policy can be used:

{
  "Version": "2012-10-17",
  "Id": "read-only-subdirectory",
  "Statement": [
    {
      "Sid": "project-list_bucket",
      "Effect": "Allow",
      "Principal": {
       "CanonicalUser": "READ_ONLY_CLOUDIAN_CANONICAL_ID"
      },
      "Action": [
       "s3:ListBucket"
      ],
      "Resource": [
"arn:aws:s3:::SHARED_BUCKET",
       "arn:aws:s3:::SHARED_BUCKET/*"
      ]
    },
    {
     "Sid": "project-read_subdirectory",
     "Effect": "Allow",
     "Principal": {
      "CanonicalUser": "READ_ONLY_CLOUDIAN_CANONICAL_ID"
},
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::SHARED_BUCKET/foo/bar/*"
]
}
]
}
  • the project-list_bucket statement is not mandatory, but without it the read only project can not list the bucket's objects, and therefore have to know the object path in advance.
  • READ_ONLY_CLOUDIAN_CANONICAL_ID is the principle identifier of the project with read only access to the bucket.
  • SHARED_BUCKET/foo/bar/* is the bucket/objects sub-directory prefix to restrict access to.
  • the element "Version": "2012-10-17" is mandatory.
{
  "Version": "2012-10-17",
  "Id": "read-only-subdirectory",
  "Statement": [
    {
      "Sid": "project-list_bucket",
      "Effect": "Allow",
      "Principal": {
       "AWS": "arn:aws:iam::READ_ONLY_PROJECT_ID:root"
      },
      "Action": [
       "s3:ListBucket"
      ],
      "Resource": [
       "arn:aws:s3:::*"
      ]
    },
    {
     "Sid": "project-read_subdirectory",
     "Effect": "Allow",
     "Principal": {
      "AWS": "arn:aws:iam::READ_ONLY_PROJECT_ID:root"
},
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::SHARED_BUCKET/foo/bar/*"
]
}
]
}
  • the project-list_bucket statement is not mandatory, but without it the read only project can not list the bucket's objects, and therefore have to know the object path in advance.
  • READ_ONLY_PROJECT_ID is the principle identifier of the project with read only access to the bucket.
  • SHARED_BUCKET/foo/bar/* is the bucket/objects sub-directory prefix to restrict access to.
  • the element "Version": "2012-10-17" is mandatory. 

Then the bucket's owner uses the s3cmd command to apply and check the policy on the bucket:

s3cmd -c s3cfg-OWNER_PROJECT setpolicy read-only-subdirectory.json s3://SHARED_BUCKET
s3cmd -c s3cfg-OWNER_PROJECT info s3://SHARED_BUCKET
Policy: Read and Write Bucket and Objects

If the project owner (e.g. OWNER_CLOUDIAN_CANONICAL_IDOWNER_PROJECT_ID) of the bucket (e.g. SHARED_BUCKET) want to share it read and write with another project (e.g. READ_WRITE_CLOUDIAN_CANONICAL_ID, READ_WRITE_PROJECT_ID), the following policy can be used:

{
"Version": "2012-10-17",
"Id": "read-write",
"Statement": [
{
"Sid": "project-read_write",
"Effect": "Allow",
"Principal": {
"CanonicalUser": [
"OWNER_CLOUDIAN_CANONICAL_ID",
"READ_WRITE_CLOUDIAN_CANONICAL_ID"
]
},
"Action": [
"s3:ListBucket",
"s3:PutObject",
"s3:DeleteObject",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::SHARED_BUCKET",
"arn:aws:s3:::SHARED_BUCKET/*"
]
}
]
}
  • OWNER_CLOUDIAN_CANONICAL_ID is the principle identifier of the project sharing the bucket. The bucket owner must be listed explicitly in the Principal AWS list.
  • READ_WRITE_CLOUDIAN_CANONICAL_ID is the principle identifier of the project with read and write access to the bucket.
  • the element "Version": "2012-10-17" is mandatory. 
{
"Version": "2012-10-17",
"Id": "read-write",
"Statement": [
{
"Sid": "project-read_write",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::OWNER_PROJECT_ID:root",
"arn:aws:iam::READ_WRITE_PROJECT_ID:root"
]
},
"Action": [
"s3:ListBucket",
"s3:PutObject",
"s3:DeleteObject",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::*"
]
}
]
}
  • OWNER_PROJECT_ID is the principle identifier of the project sharing the bucket. The bucket owner must be listed explicitly in the Principal AWS list.
  • READ_WRITE_PROJECT_ID is the principle identifier of the project with read and write access to the bucket.
  • the element "Version": "2012-10-17" is mandatory. 

Then the bucket's owner uses the s3cmd command to apply and check the policy on the bucket:

s3cmd -c s3cfg-OWNER_PROJECT setpolicy read-write.json s3://SHARED_BUCKET
s3cmd -c s3cfg-OWNER_PROJECT info s3://SHARED_BUCKET