#Ingest Rules

An ingest rule is a standing instruction on one storage origin: when an object matching these filters appears, create this job for it. Your storage provider posts an event to the rule’s endpoint, we record it, and a matching event becomes a job — with no server of yours in the path.

object lands in your bucket
        │
        ▼
provider notification  ──POST──►  /ingest/{ing_id}
                                      │  authenticate, parse, deduplicate
                                      ▼
                                 storage event          received
                                      │
                                      ▼
                            filters, then CreateJob
                                      │
                                      ▼
                                 storage event          created · skipped · failed

A rule belongs to an app and watches exactly one origin. The origin must be active and have read permission — checked when the rule is created, not at first delivery, because a rule on a write-only origin would accept events forever and never fetch a byte.

#Creating a rule

curl -X POST https://api.transcodely.com/transcodely.v1.IngestRuleService/Create 
  -H "Authorization: Bearer {{API_KEY}}" 
  -H "Content-Type: application/json" 
  -d '{
    "origin_id": "ori_a1b2c3d4e5f6",
    "name": "Inbox uploads",
    "filters": {
      "prefix": "uploads/",
      "suffixes": [".mp4", ".mov"],
      "min_bytes": 1024
    },
    "action": {
      "outputs": [{ "preset": "pst_x9y8z7w6v5" }],
      "managed": true,
      "output_path_template": "{input_dir}/{input_name}/{job_id}"
    }
  }'

The response carries the inbound secret in full, once:

{
  "rule": {
    "id": "ing_a1b2c3d4e5f6",
    "app_id": "app_k1l2m3n4o5",
    "origin_id": "ori_a1b2c3d4e5f6",
    "name": "Inbox uploads",
    "enabled": true,
    "endpoint_url": "https://api.transcodely.com/ingest/ing_a1b2c3d4e5f6",
    "secret_prefix": "ings_a1b",
    "secret_hint": "z9y8",
    "events_received": 0,
    "jobs_created": 0
  },
  "secret": "ings_a1b2c3d4e5f6g7h8i9j0z9y8"
}

We keep only enough of the secret to verify a delivery and to show you its first and last characters, so it cannot be recovered later. If you lose it, rotate the rule with Update and rotate_secret — the previous secret keeps verifying for 24 hours so you can update the sender without dropping an event.

#The endpoint

POST https://api.transcodely.com/ingest/{ing_id}

Three proofs are accepted, because the senders are not equally capable. Use the strongest one yours can manage.

Signature — best. It proves possession of the secret without transmitting it, and it bounds replay:

Transcodely-Signature: t=<unix>,v1=<hex>

v1 is HMAC-SHA256(secret, "<t>.<raw body>"), hex-encoded — the same scheme and the same function that signs outbound webhooks, so a verifier you already wrote signs with no changes. Several v1 entries are legal, which is what makes rotation work. The window is five minutes. The older X-Transcodely-Signature spelling is still accepted.

Bearer — what a sender that can set one header can attach:

Authorization: Bearer ings_…

Query token — the floor, for senders that can set no header at all:

POST /ingest/ing_a1b2c3d4e5f6?token=ings_…

An Amazon SNS HTTPS subscription and a Google Pub/Sub push subscription both fall here: neither can attach an Authorization header of yours. Without this form they would be refused on every delivery, including the SNS confirmation handshake, so the subscription would never begin working and the only symptom would be silence.

What it costs, plainly. A URL travels through more places than a header. Proxies and gateways log request lines, browsers keep history, consoles display it back to whoever can see the page, and it is the value you paste into someone else’s form. Treat a ?token= secret as more exposed than a header one, and rotate it if the URL is ever shared. We never log it, never echo it in an error, and never return it.

Rotation covers all three forms: the previous secret keeps verifying for 24 hours, which is exactly what the pasted-into-a-console case needs.

#Responses

The success body is small and stable:

{ "id": "sev_a1b2c3d4e5f6g7", "status": "received" }

status is received for a new event, duplicate for one already recorded, confirmed for a completed subscription handshake, and ignored for a payload we understood and deliberately did not act on. A delivery that names several objects also carries events with every id.

Nothing touches your rule’s budget until the secret verifies. A rule id is not a secret — it is the path of a URL you paste into a console, and it comes back on every read — so if a refused request spent a token from the rule’s own bucket, anyone holding that URL could hold it at zero with empty posts and your real deliveries would get 429. SNS would retry and survive that; a Supabase webhook and a Pub/Sub push would not, and those events would be lost.

#Payload shapes

The shape is detected from the payload. You never declare it.

The generic shape also accepts object_key and name for key, eTag for etag, and size_bytes for size.

Amazon S3 percent-encodes object keys and writes a space as +. We decode them: the stored key, and the key the job reads, is the real one. For Google Cloud Storage only OBJECT_FINALIZE is acted on, and the object generation is its version identity. For Supabase the trigger must be INSERT only — an upload also writes an UPDATE once its metadata lands, and the two carry different eTags, which would give one upload two jobs.

#Which auth form each sender uses

The dashboard renders each of these against your rule, on the origin’s page.

#Filters

All set filters must pass. An empty filter set matches everything in the bucket.

An object that fails a filter is recorded as skipped, never as an error.

#The action

The action is the part of a job that can be decided ahead of time: outputs, managed or output_origin_id, output_path_template, thumbnails, priority and metadata. The input is what the event supplies — the rule’s origin, and the key the event names.

Set exactly one output destination: managed for Transcodely hosting, or output_origin_id for one of your own write-enabled origins.

Beyond the job path variables, a rule’s output_path_template accepts three more, so outputs can mirror the source layout:

Provider-supplied keys are sanitized before substitution: a leading / and any .. segment are stripped, so a crafted object key cannot write outputs outside the prefix you configured.

Every job a rule creates carries ingest_rule_id and ingest_object_key in its metadata, alongside any metadata you set on the action.

#Deduplication

Every one of the supported sources delivers at least once, and some deliver the same object twice on purpose. An object is identified by (rule, bucket, key, etag), and a redelivery is answered 202 with the id of the event that already won.

The job carries a second, independent guard: its idempotency key is derived from the same tuple, hashed, because an object key alone can run to 1024 bytes and the key field is capped at 128 characters. Together these mean a retry produces the same single job.

A new version of the same key is a different object and gets its own job — when the source reports a version identity. S3, R2 and Google Cloud Storage always do: an eTag or a generation.

When it does not, a re-upload over the same key is treated as a duplicate and produces no job, because the eTag is stored empty and the tuple is unchanged. This affects Supabase most: its INSERT often carries no eTag at all, and the re-upload itself arrives as an UPDATE, which is ignored by design. ReplayEvent on the original event is the remedy — it re-runs the rule against whatever now lives at that key.

#What happens to an event

{
  "id": "sev_a1b2c3d4e5f6g7",
  "rule_id": "ing_a1b2c3d4e5f6",
  "app_id": "app_k1l2m3n4o5",
  "bucket": "customer-uploads",
  "object_key": "uploads/2026/holiday.mp4",
  "etag": "d41d8cd98f00b204e9800998ecf8427e",
  "size_bytes": 734003200,
  "content_type": "video/mp4",
  "source": "generic",
  "status": "created",
  "job_id": "job_a1b2c3d4e5f6"
}

reason is a short slug, and it is part of the contract:

A paused rule still answers 202 and records the event as skipped, so a provider subscription does not start failing and backing off while ingestion is switched off. Those objects are not transcoded when you turn the rule back on — nothing retries them on its own. Turning a rule back on returns events_skipped_while_disabled, the count of objects waiting for you, and ReplayEvent is how you act on them.

A definitive refusal is never retried: an app over its monthly spend cap stays over it until the cap or the month changes, so retrying the same object would only fill the log. A transient failure on our side backs off and is retried for up to five attempts before it becomes a failure.

#Managing rules

Deleting a rule is a soft delete: the endpoint stops immediately and the events stay readable, because “why was this file never transcoded” must not be answered by a table that forgot.

#Replaying events

Deduplication is permanent. An object is (rule, bucket, key, etag), so re-sending the event — or re-uploading the same bytes — is absorbed and produces nothing. Without a replay, anything that reached skipped or failed would be un-ingestable forever: everything that arrived during a pause, everything refused while an account was over its spend cap, everything that exhausted its transient attempts.

curl -X POST https://api.transcodely.com/transcodely.v1.IngestRuleService/ReplayEvent 
  -H "Authorization: Bearer {{API_KEY}}" 
  -H "Content-Type: application/json" 
  -d '{ "event_id": "sev_a1b2c3d4e5f6g7" }'

That resets that row back to received with its attempt count cleared. The row is reset rather than duplicated, so the event keeps its id and its history and the deduplication tuple still holds.

Only skipped and failed can be replayed. One still queued would be queued twice, and one that already created a job has nothing to redo — both answer failed_precondition with storage_event_not_replayable. In the dashboard the control only appears on the rows the API will accept.

#Testing a rule before you wire anything

Test answers whether a key would match and, when it would, returns the exact job request the rule would submit. Nothing is stored and no job is created.

curl -X POST https://api.transcodely.com/transcodely.v1.IngestRuleService/Test 
  -H "Authorization: Bearer {{API_KEY}}" 
  -H "Content-Type: application/json" 
  -d '{
    "id": "ing_a1b2c3d4e5f6",
    "object_key": "uploads/2026/holiday.mp4",
    "size_bytes": 734003200,
    "content_type": "video/mp4",
    "etag": "d41d8cd98f00b204e9800998ecf8427e"
  }'

The etag changes no filter decision. It is what the previewed idempotency key is derived from, so supplying the one a real delivery will carry makes the preview name the key the job will actually use.

#Next

• Supabase Storage upload to HLS
• Cloudflare R2 to HLS
• Origins — the bucket a rule watches
• Presets — what a rule’s outputs reference