Data Export to Amazon S3: permissions step by step
How to prepare an S3 bucket and give AdShift a cross-account IAM role you can revoke and audit yourself, rather than an access key pair.
Amazon S3 receives partitioned Parquet or CSV files, copied into a bucket you own. AdShift needs permission to write there, and there are two ways to give it: a cross-account IAM role, or an access key pair.
Prefer the role. You can revoke it, audit every use of it in CloudTrail, and there is no long-lived secret sitting in our database. The key pair exists for organisations whose security review cannot accommodate a cross-account trust policy.
This article is the permissions walkthrough for Amazon S3 exports.
What you need
- An AWS account where you can create a bucket and an IAM role.
- AdShift's export role ARN, which support provides for your environment. It looks like
arn:aws:iam::<AdShift account>:role/prod-data-export-runner-role. - An external ID. You invent this one — any unguessable string, e.g. a UUID. You will put the same value in the trust policy and in the connection.
- About ten minutes.
1. Create a dedicated bucket or prefix
Use a bucket, or at least a prefix, only for AdShift data. Exported partitions are written and, on a re-delivery, written again under a new version folder.
In the console: S3 → Create bucket. Block all public access (the default), and pick the region closest to whatever will read the data — the region goes into the connection, and cross-region reads cost you egress.
If you want the data under a folder rather than at the bucket root, decide the prefix now, e.g. adshift. You will enter it in the connection.
2. Option A (recommended): a cross-account role
2.1 Create the policy
IAM → Policies → Create policy → JSON. Replace YOUR-BUCKET and, if you use one, YOUR-PREFIX:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "WriteExportedPartitions",
"Effect": "Allow",
"Action": ["s3:PutObject", "s3:GetObject", "s3:DeleteObject"],
"Resource": "arn:aws:s3:::YOUR-BUCKET/YOUR-PREFIX/*"
},
{
"Sid": "ListForCleanRetries",
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::YOUR-BUCKET",
"Condition": {
"StringLike": { "s3:prefix": "YOUR-PREFIX/*" }
}
}
]
}All four actions are used by a normal delivery: we write the partition, read a blob back during the connection test, list a prefix so a retry is clean, and delete the probe object the test leaves behind. A policy granting PutObject alone passes the first check and fails afterwards.
If you write at the bucket root, drop YOUR-PREFIX/ from the resource and remove the condition.
2.2 Create the role
IAM → Roles → Create role → Custom trust policy. Paste this, substituting AdShift's role ARN and the external ID you chose:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": { "AWS": "ADSHIFT-EXPORT-ROLE-ARN" },
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": { "sts:ExternalId": "YOUR-EXTERNAL-ID" }
}
}
]
}Attach the policy from 2.1 and name the role, e.g. adshift-data-export.
Do not skip the external ID. Without it, anyone who learns the role ARN and can reach AdShift's account could ask us to assume your role. The condition is what makes that impossible — this is the confused-deputy problem AWS documents for exactly this pattern.
2. Option B: an access key pair
Create an IAM user with no console access, attach the same policy from 2.1, and create an access key under Security credentials.
The secret key is stored encrypted and is never shown back to you; the connection form displays [redacted] and leaving the field blank on an edit keeps the stored value. Rotating it means creating a new key and pasting it into the connection.
3. Connect it in AdShift
Go to Data Export → Connections → New connection, pick S3 and fill in:
| Field | What to enter |
|---|---|
| Connection name | Lowercase letters, digits and hyphens, e.g. s3-marketing |
| S3 bucket | The bucket name only, without s3:// |
| AWS region | The bucket's region, e.g. eu-central-1 |
| Path prefix | Optional, e.g. adshift. Leave empty to write at the bucket root |
| Authentication | Assume IAM role (recommended) or Access key pair |
| IAM role ARN + External ID, or Access key ID + Secret access key | From step 2 |
The panel shows a live preview of where data will land as you type.
Click Test connection. For the role path, the first check is the role assumption itself — reported separately, because a trust-policy problem is a different thing to fix than a bucket-policy problem. Then a probe object is written, read back and deleted, each reported on its own line.
Then click Save, open Reports on the connection, and pick your datasets.
What you get in the bucket
s3://your-bucket/
adshift/ <- your path prefix, if you set one
t=clicks/dt=2026-09-15/h=14/
data_0_0_0.snappy.parquet
_SUCCESS
t=is the dataset,dt=the date,h=the hour. Daily datasets have noh=element.- The layout is Hive-style, so Athena, Glue and Spark discover the partitions without extra configuration. In Athena, point an external table at the prefix and run
MSCK REPAIR TABLE. _SUCCESSis written after every data file. Wait for it before reading a folder; a folder without it is still being written.- A re-delivery after late data adds a
v=2element (thenv=3, and so on). The first delivery of a partition has no version element. Read the highest version present and treat earlier ones as superseded. - If you chose By app folder structure, an
app_id=element appears before the version.
If something goes wrong
The sts:AssumeRole check fails — the trust policy is the problem, not the bucket. Check that the principal is AdShift's export role ARN exactly, and that the external ID in the trust policy and in the connection match character for character.
AccessDenied on s3:PutObject — the resource ARN in the policy does not cover the prefix you entered in the connection. arn:aws:s3:::bucket/adshift matches nothing; it needs the trailing /*.
AccessDenied on s3:DeleteObject only — a common policy trims delete for safety. AdShift needs it to clear a partial partition before a retry; without it a retried delivery mixes files from two attempts.
KMS.AccessDeniedException — the bucket uses SSE-KMS with a customer-managed key. Add kms:GenerateDataKey and kms:Decrypt on that key to the policy, and allow the role in the key policy.
Files appear but your query returns nothing — partitions have not been discovered. Run MSCK REPAIR TABLE in Athena, or check that the table location matches the prefix including any folder you set.
Revoking access
Disable or delete the connection in AdShift first, then delete the role or the access key. Doing it the other way round just makes the next delivery fail with a permission error. Data already delivered is in your bucket and is unaffected either way.
Network Self-Reports
Network Self-Reports shows performance data as reported directly by each ad network — spend, impressions, and campaign metrics from Meta, Apple, and TikTok.
Data Export to Azure Blob Storage: setup step by step
How to create a container, issue a SAS token with the four permissions a delivery needs, and connect it in AdShift.