+1 (726) 227-3497
RougeWarehouse

Governing Amazon Redshift Through SageMaker Lakehouse and Lake Formation

Amazon Redshift is no longer just a cluster you point a BI tool at. With SageMaker Lakehouse and the Amazon SageMaker Unified Studio, AWS folded Redshift, Glue catalogs, S3 tables and Athena engines into one catalog-and-permissions plane governed by AWS Lake Formation. For teams that already run Redshift, the practical question is narrow: how do you expose your existing warehouse through the lakehouse catalog without loosening the access controls you spent a year building?

This tutorial walks that path end to end: registering a Redshift namespace as a lakehouse catalog, granting fine-grained Lake Formation permissions, querying the same tables from Redshift and from Spark, and the gotchas that come from having two permission systems in front of one table. Console flows move around between releases, so treat the click paths as orientation and the SQL/CLI as the durable part.

The mental model: three layers, one catalog

  • Storage - Redshift managed storage (RA3 or Serverless) and S3, including Iceberg and S3 Tables.
  • Catalog - SageMaker Lakehouse presents both as catalogs in Glue. A Redshift namespace appears as a federated catalog whose databases and schemas are addressable from outside Redshift.
  • Permissions - Lake Formation grants (database, table, column, row) apply to consumers arriving through the catalog. Redshift's own GRANT/RBAC still governs sessions that arrive over JDBC to the warehouse.

That last split is the most important thing to internalise. Registering the warehouse does not delete or replace your in-database grants. It adds a second door with its own lock.

Step 1: prerequisites you actually need

  1. A Redshift Serverless workgroup or an RA3 provisioned cluster in the same account and region as the Glue Data Catalog you intend to use.
  2. Lake Formation configured with at least one data lake administrator principal. If Lake Formation is brand new in the account, databases may still be in "use only IAM access control" mode; you have to clear that for anything you want Lake Formation to govern.
  3. An IAM role Redshift can assume for catalog and S3 access, attached to the namespace:
aws redshift-serverless update-namespace \
  --namespace-name analytics \
  --iam-roles arn:aws:iam::111111111111:role/RedshiftLakehouseRole \
  --default-iam-role-arn arn:aws:iam::111111111111:role/RedshiftLakehouseRole

The role needs Glue read actions on the catalog, lakeformation:GetDataAccess, and S3 read on the lake buckets. Start read-only; add write actions only for catalogs you genuinely intend to write to.

Step 2: register the Redshift namespace with the lakehouse catalog

In the Lake Formation console this is Data catalog -> Catalogs -> Create catalog, source type Redshift, pointing at your namespace or workgroup, with an IAM role for the catalog to use. What it produces is a catalog entry whose identifier looks like <account-id>:<redshift-database> - the multi-level catalog naming the lakehouse uses to distinguish a federated Redshift database from a plain Glue database.

Confirm it landed:

aws glue get-catalogs --query 'CatalogList[].{Name:Name,Type:CatalogType}'
aws glue get-databases --catalog-id 111111111111:analytics \
  --query 'DatabaseList[].Name'

If the catalog exists but get-databases comes back empty, the usual cause is the catalog's IAM role lacking permission on the Redshift namespace - not a missing Lake Formation grant.

Step 3: grant fine-grained access in Lake Formation

Grant a consumer principal access to one table, and only some of its columns:

aws lakeformation grant-permissions \
  --principal DataLakePrincipalIdentifier=arn:aws:iam::111111111111:role/AnalystSpark \
  --resource '{
      "TableWithColumns": {
        "CatalogId": "111111111111:analytics",
        "DatabaseName": "finance",
        "Name": "gl_entries",
        "ColumnNames": ["posted_on","cost_center_id","amount"]
      }
   }' \
  --permissions SELECT

Row-level filtering uses a data cells filter rather than a column list:

aws lakeformation create-data-cells-filter --table-data '{
  "TableCatalogId": "111111111111:analytics",
  "DatabaseName": "finance",
  "TableName": "gl_entries",
  "Name": "emea_only",
  "RowFilter": {"FilterExpression": "region = '"'"'EMEA'"'"'"},
  "ColumnWildcard": {"ExcludedColumnNames": ["employee_id"]}
}'

Then grant SELECT on the filter instead of the table. This is how you serve one physical table to several teams without maintaining a per-team view layer.

Step 4: query from both doors

From Redshift (the JDBC door), lakehouse databases are reachable through an external schema:

CREATE EXTERNAL SCHEMA lake_finance
FROM DATA CATALOG
DATABASE 'finance'
CATALOG_ID '111111111111'
IAM_ROLE 'arn:aws:iam::111111111111:role/RedshiftLakehouseRole';

SELECT cost_center_id, SUM(amount) AS total
FROM lake_finance.gl_entries
WHERE posted_on >= DATE '2026-01-01'
GROUP BY 1
ORDER BY 2 DESC;

Joining warehouse-resident and lake-resident tables in one statement is the whole point. The optimizer pushes down predicates and projections for Parquet and Iceberg scans, so keep partition and date predicates in the query rather than buried in a wrapper view.

From Spark in Unified Studio, the same table is reachable through the catalog name:

spark.sql("""
  SELECT cost_center_id, SUM(amount) AS total
  FROM `111111111111:analytics`.finance.gl_entries
  WHERE posted_on >= '2026-01-01'
  GROUP BY 1
""").show()

If Spark returns three of the table's twenty columns, that is not a bug - that is the column grant from step 3 doing its job.

Step 5: keep governance auditable

  • Write grants as code. Lake Formation grants are as security-relevant as IAM policies and just as easy to accumulate by hand. Keep grant-permissions in Terraform (aws_lakeformation_permissions) beside the namespace definitions, the same way you manage the rest of the warehouse as code.
  • Prefer LF-Tags past a few dozen tables. Tag finance schemas Confidentiality=High, grant roles on the tag, and new tables inherit instead of needing a new grant.
  • Log both doors. Catalog access shows up in CloudTrail (GetDataAccess); in-warehouse access shows up in SYS_QUERY_HISTORY and the connection log. A compliance answer needs both.
  • Decide where each control lives. Redshift RBAC, RLS and dynamic data masking still apply to JDBC sessions. Duplicating the same rule in Lake Formation and in the database is how you end up with two rules that disagree six months later.

Gotchas worth budgeting time for

  • Region and account alignment. Federated Redshift catalogs are happiest in-region and in-account. Cross-account topologies are usually better served by plain Redshift data sharing, which has a simpler permission story.
  • DC2 cannot participate. Anything lakehouse-adjacent assumes RA3 or Serverless. If you are still on DC2, migrate first.
  • The IAMAllowedPrincipals default. A database created before Lake Formation governance was enabled can still carry it, which effectively grants everyone and makes your careful column grant look like it did nothing. Remove it explicitly.
  • Write paths lag read paths. Validate any Spark-writes-to-Redshift or Redshift-writes-to-Iceberg flow against your own volumes before you design an ELT pipeline around it.
  • Cost attribution. A Spark query against a federated Redshift catalog consumes Redshift compute. If chargeback matters, keep one workgroup per consuming team so the RPU-hours land on the right tag.

A sensible rollout order

  1. Register one low-risk schema as a catalog. Nothing else.
  2. Grant a single read-only role with an explicit column list, and prove the restriction from Spark.
  3. Move those grants into Terraform, then introduce LF-Tags.
  4. Only then expand to sensitive schemas - and write down which control lives where.

Done in that order, the lakehouse catalog is an additive change: analysts and BI keep their existing JDBC path into Redshift, and Spark or Athena consumers get governed, column- and row-filtered access to the same tables with no copies and no nightly export.

If you want the catalog layout, the Lake Formation permission model and the split between catalog and in-database controls designed before any of it touches production data, that is the work our Redshift consultants and Data Modeling & Architecture practice does. Get in touch with the schemas you want to expose and who needs to read them.