You can protect your S3 buckets using Cyral. See Track an S3 storage location for set-up instructions.
The article below is a supplement to the main Cyral documentation and explains advanced configuration settings.
Track the S3 endpoint in Cyral
In the Cyral management console, navigate to the Data Repos tab and click the plus button.
In the pop-up dialog, enter Amazon S3 as the Repository Type, and click Track.
Install the sidecar
If you don't already have a sidecar deployed that can serve this repository, add it now as shown in Install a sidecar.
Associate the S3 endpoint with your sidecar
To protect a repository and allow users to connect to it, you must associate the tracked repository with its sidecar.
In the Sidecars tab, select the sidecar to which you'd like to assign the repository and click the plus sign.
In the Assign a Repository window, choose the name of the S3 repository you created above, and specify the hostname and port where data users will connect to this repository. The TLS toggle is always set to ON for S3 repositories.
In the pop-up dialog, select your repository and click Track.
Your S3 data endpoint is now accessible through the Cyral sidecar. See the next section for connection instructions.
Connect to an S3 data endpoint through Cyral
For connecting to your S3 endpoint through the sidecar, we need to take the following steps:
Get the sidecar endpoint
Go to the Cyral Control Plane and find the sidecar endpoint associated with the S3 data repository. To do this, navigate to the Sidecars page, click the name of the sidecar, and note the Sidecar Endpoint for
In the above figure, the host is www.sidecar-endpoint.com and the port is 453.
Set the sidecar endpoint as the HTTP proxy address
In a shell session, we need to export the following environment variables:
and if the host machine is a AWS EC2 instance, you also need to export the variable:
Detailed information and settings for configuring the proxy endpoint for different systems is available at: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-proxy.html
If you're using Nifi to access S3, see Using NiFi to access data in a Cyral-protected S3 bucket.
Get an SSO token and add it to your credentials file
Append the following profile information to the AWS credentials file. This file is usually found at ~/.aws/credentials .
userA01@company.com is the email address used to log in to the Control Plane.
c17c7b65c159544f75bd98e91d584 is the access token copied from the Control Plane. You can also get the token from the Cyral CLI token retriever for SSO, gimme_db_token. For instructions, see Connect to S3 data using the CLI token retriever.
Access S3 using the profile you created
Run S3 commands specifying the previously created sidecar profile:
This uses the credentials collected from the Control Plane and validates the user, sending all traffic through the sidecar. Logs should be available at the log location configured in your Cyral installation, such as Cloudwatch, ELK, or Splunk.
Alternative and extra configurations
Adding a certificate authority bundle to the CLI tool
The sidecar needs to intercept the TLS communication between the user and AWS servers. For keeping the connection secure, the sidecar signs the messages using its own certificate. Some CLI clients and tools may complain about the certificate used by the sidecar, so an extra configuration needs to be performed:
First, download the certificate bundle from the control plane using the following command:
Next, provide the CA bundle to the AWS CLI using one of the following commands.
Using an environment variable:
cyral_ca_bundle.pem: This file is used by the AWS CLI tool to validate the certificate sent by the sidecar. This file will be shared by the Cyral’s Support team.
Using profile configuration: Append the following information to the configuration file (the location will be either or defined by the environment variable)
Both options above (a) and (b) are equivalent and only one needs to be performed
Using a profile configuration instead of environment variables for proxy settings
Throughout this guide, we've used environment variables to configure the proxy settings used by the AWS CLI tool. An alternative approach is to add the proxy settings to the configuration profile instead. To do this, we need to add a third-party plugin, "awscli-plugin-s3-proxy", to the AWS CLI tool.
For the steps below, we are assuming the profile sidecar_sample_profile will be used.
|Note: If your system doesn't recognize thecommand, you can try one of the following:|
1. Install the plugin, "awscli-plugin-s3-proxy", using python pip:
python -m pip install awscli-plugin-s3-proxy --user
2. If you are using the AWS CLI v2 (aws --version), you will need to define the appropriate location for the plugin:
a. Get the plugin location:
python -m pip show awscli-plugin-s3-proxy
It is expected to have a Location key in the generated output, for example:
- Linux: Location: /home/<username>/.local/lib/python3.8/site-packages
- Mac: Location: /Users/<username>/.local/lib/python3.8/site-packages
- Windows: Location: c:\Users\<username>\Application Data\python\python38\site-packages
b. Set the plugin location with the command:
aws configure set plugins.cli_legacy_plugin_path <LOCATION>
which for our Mac output above, would be, for example:
aws configure set plugins.cli_legacy_plugin_path /Users/<username>/.local/lib/python3.8/site-packages
3. Configure using the AWS CLI:
aws configure set plugins.s3-proxy awscli_plugin_s3_proxy
4. Add the sidecar as a proxy for S3:
aws configure --profile sidecar_sample_profile set s3.proxy http://<YOUR_SIDECAR_ADDRESS>:453
replacing <YOUR_SIDECAR_ADDRESS> with the actual address.
5. Add the Cyral CA bundle to the same profile:
aws configure --profile sidecar_sample_profile set ca_bundle /path/to/cyral_ca_bundle.pem
Enable SSO login on your S3 data endpoint
To enable SSO login on your S3 data repository, select the S3 repository and go to the Advanced tab. Here, make the following settings.
Choose your Identity Provider from the drop-down box.
Unselect the checkbox Allow native authentication.
Under Enforcement, enable Block on violations.
Provide the IAM roles needed for accessing S3
Under “Local Accounts'' tab for the S3 repo specify the IAM roles that provide various levels of access to S3. Depending on which end user is connecting, the sidecar will assume one of these roles and make the S3 request using the local account assigned to that user. For more information on IAM role settings see "Make AWS IAM role settings” at the end of this section
In the Cyral control plane UI, go to Repositories, click Local Accounts, and click Track Account.
In the Track Account form, enter the IAM role ARN
In the below screenshot a role providing full s3 access is shown as example
Map an SSO user or group to the IAM Role
When a user authenticates, they can be mapped to use a specific IAM role to access S3 based on their username, or based on their membership in an SSO group. Set up the mapping as follows.
In the Repositories page, click Identity to Account Map and click the plus sign.
Choose User or Group as the identity type.
In the Identity field, specify the SSO user name or group name as it's written in your identity service.
In the Local Account field, choose the name of the IAM role
In the Duration field, set a length of validity for the access, or click Unlimited to grant access that will not expire automatically.
In the below screenshot users belonging to “Data Analysts” SSO group are mapped to user “SidecarS3FullAccess” role
Make AWS IAM role settings
It is important to make sure that the IAM role associated with the sidecar is trusted by the IAM roles used for managing S3.
Find your sidecar host role
After deploying the sidecar, the IAM role associated with the sidecar is created with name <sidecar-cft-stack-name>SidecarHostRole-*
Below is an example sidecar host role created for a sidecar with stack name “jc-t01”
Create suitable IAM roles for S3 access and establish trust with sidecar role
The various roles and the extent of permissions the roles get is up to the organization. In the below example, we have show two roles “SidecarReadOnlyRole” and “SidecarS3FullAccess”. There is a trust relationship between these roles and the SidcarHostRole, which allows sidecar to assume these roles.
By having the sidecar role as a trusted entity for your S3 management roles, you are allowing the sidecar to assume these custom roles, when required, for connecting to the S3 servers when handling SSO connections.
This information can also be edited in the AWS Console in the IAM → Roles section.
Users can connect to S3 using the CLI token retriever for SSO
Data users can use the Cyral CLI token retriever for SSO, gimme_db_token, to quickly authenticate and connect to their S3 data endpoints. For instructions, see Connect to S3 data using the CLI token retriever.