← SPICE Internals and Data at Scale

Runbook — SPICE ingestion failures

For someone under pressure. No background. Teaching lives in the lessons; this page is symptom → command → decision.

export ACCOUNT=<12-digit account id>
export REGION=<the Quick Sight account's region>
export DATASET=<dataset id>

0. The one command

aws quicksight list-ingestions --aws-account-id $ACCOUNT --data-set-id $DATASET \
  --region $REGION --max-results 20 \
  --query 'Ingestions[].{When:CreatedTime,Status:IngestionStatus,Src:RequestSource,
      Type:RequestType,Secs:IngestionTimeInSeconds,In:RowInfo.RowsIngested,
      Dropped:RowInfo.RowsDropped,Err:ErrorInfo.Type}' --output table

Wrong Region ⇒ ResourceNotFoundException. Check the Region before you check anything else.

---

Symptom: dashboard is stale

---

Symptom: FAILED — route by ErrorInfo.Type

Full enum: ErrorInfo

---

Symptom: dropped rows on a COMPLETED ingestion

1. Record RowsIngested, RowsDropped, TotalRowsInDataset and the timestamp.
2. Look for DATA_TOLERANCE_EXCEPTION, INVALID_DATE_FORMAT, SQL_NUMERIC_OVERFLOW in nearby runs — the same bad data usually shows up as an error type before or after.
3. Tell the data consumers immediately. A dashboard reporting confidently wrong numbers is worse than a broken one, and someone is currently making a decision on it.
4. Then find the offending rows at source.

Prevention: alert on RowsDropped > 0, not only on IngestionStatus == FAILED.

---

Symptom: create-ingestion fails

Check remaining budget:

aws quicksight list-ingestions --aws-account-id $ACCOUNT --data-set-id $DATASET \
  --region $REGION --max-results 100 \
  --query 'Ingestions[].RequestSource' --output text | tr '\t' '\n' | sort | uniq -c

---

Symptom: source database is being hammered by Quick Sight

Direct query only. The 2-minute visual timeout does not stop the query on Redshift — it keeps running and holds a WLM slot.

1. Cancel the running queries on the cluster.
2. Put Quick Sight in a dedicated WLM queue with a statement timeout.
3. Longer term: move the aggregate workload to SPICE and keep direct query for drill-through only.

---

Escalating to AWS Support

Collect first:

aws quicksight list-ingestions  --aws-account-id $ACCOUNT --data-set-id $DATASET --region $REGION --max-results 100 > ingestions.json
aws quicksight describe-data-set --aws-account-id $ACCOUNT --data-set-id $DATASET --region $REGION > dataset.json
aws service-quotas list-service-quotas --service-code quicksight --region $REGION > quotas.json

Case body, in order: account ID · Region · dataset ID · ingestion ARN · ErrorInfo.Type verbatim · RequestId · UTC timestamps · RowInfo · what you ruled out and the commands you ran · one question with a factual answer.

Case type: Service limit increase only for quotas marked adjustable. Ingestion failures and INTERNAL_SERVICE_ERROR are Technical. Capacity charges are Account and billing.

Severity: stale dashboard = normal (12 h) or high (4 h) if genuinely business-critical. Not urgent. You can raise it later on Business Support+, Enterprise Support, or Unified Operations — but after raising to urgent or critical there's a 60-minute wait before the next change.

If the first reply is a doc link you've already read: name the page and the line, and state what remains unanswered. Reply on the existing case; never open a duplicate.

---

Escalation contacts

Fill this in for your organisation and keep it here.