How to develop an ingest service

Goal

Build a backend service that reads content from your source system, translates it into AXIS ingestion payloads, submits it to axis-api-ingestion, and verifies the result with the Reports API.

The service should fetch and cache required platform reference data from axis-api-pm before building payloads.

Recommended architecture

Component	Responsibility
Source connector	Reads catalog content, schedules, image references, and offer-related data from your source system
Reference data client	Fetches required reference data from axis-api-pm
Reference data cache	Stores ratings, segmentation tags, relation types, user groups, offer groups, and offer template IDs
Payload mapper	Converts source records into AXIS item or schedule payloads
Payload validator	Checks payloads against known reference data and project-specific item requirements before submission
Ingestion client	Calls axis-api-ingestion
Work queue	Controls dependency ordering, retries, and tenant-level concurrency for batch ingestion
Report checker	Uses the Reports API to inspect workflow status and failed steps
Reconciliation job	Compares source feed state with ingestion reports to find failed, partial, or missed records
Monitoring and alerting	Tracks failures, partial successes, and report results during batch runs

Add schema-aware responsibilities to the mapper and validator:

Component	Schema-Aware Responsibility
Payload mapper	Selects the correct item schema, maps source fields into the schema's core fields, fills type-specific extensions, sets parentExternalId where needed, maps images and offers, and builds relation payloads
Payload validator	Validates required fields, localized text, reference data, type-specific extension structure, parent-child dependencies, image accessibility, offer group shape, and relation payload shape before submission

Step-by-step

Step 1: Load configuration

Store environment-specific values outside the code:

• axis-api-pm base URL
• axis-api-ingestion base URL
• Tenant ID
• OAuth token endpoint
• Client ID
• Client secret
• API scope
• Tenant concurrency limit
• Retry and repair settings
• Enabled item schemas for the project
• Default culture or supported culture list
• Allowed image types
• Relation types expected by the source feed

Use separate configurations for development, staging, and production.

Step 2: Authenticate

Both axis-api-pm and axis-api-ingestion require a JWT Bearer token. Request a token from the configured OAuth2 authority using the client credentials provided by the platform team.

Authorization: Bearer {token}

Tokens are time-limited. Refresh them before expiry and do not cache them indefinitely.

If a request fails with 401, refresh the JWT and retry after refreshing the token.

The source documentation marks authentication as To Be Implemented, Skip for now. Confirm the final token endpoint, scope, and credential process with the platform team before production use.

Step 3: Warm the reference data cache

Before submitting content, fetch the required reference data from axis-api-pm.

GET /v0.1/{tenantId}/ratings
GET /v0.1/{tenantId}/segmentation-tags
GET /v0.1/{tenantId}/relation-types
GET /v0.1/{tenantId}/user-groups/lookup
GET /v0.1/{tenantId}/offer-groups
GET /v0.1/{tenantId}/offer-groups/{groupId}/offers

Data	Recommended Cache TTL
Rating systems	24 hours
Segmentation tags	1 hour
Offers	5 minutes
Relation types	1 hour
User groups	1 hour

If a validation failure mentions an unknown ID or key, re-fetch reference data before retrying.

If your item type uses custom extension properties, request the relevant JSON schema and validate the extensions object against it.

Also load or configure the item schemas your feed supports. At minimum, the ingest service should know the required fields and extension shape for each item type it maps.

Step 4: Map source records to AXIS payloads

For catalog items, payloads commonly include:

• externalId
• itemType
• Localized title
• Localized description, where required by the item schema
• Ratings
• segmentationTags
• Categories
• Keywords
• availabilityWindows
• Images
• offerGroups
• parentExternalId for child content
• extensions for custom assets

Confirm required fields for each item type with the platform team or the relevant item schema.

Step 4A: Select the correct item schema

Before mapping fields, classify each source record into an AXIS item type.

Source Record	AXIS Item Type	Mapper Responsibility
Standalone movie	Movie	Map movie title, descriptions, ratings, images, availability, offers, and movie-specific extensions such as duration, release year, cast, crew, genres, advisory, and media references.
Editorial program, documentary, highlight, replay, interview, or news item	Program	Map program metadata and program-specific extensions such as subtype, sequence number, location, venue, duration, broadcast date, event date, cast, crew, and genres.
Event-based content	Event	Map event title, timing, venue/location where available, ratings, images, availability, offers, and event-specific metadata.
Promotional short-form item	Trailer	Map trailer metadata, images, availability, offers, and destination or parent references where the project requires them.
Series container	Show	Map show-level metadata and ingest before dependent seasons or episodes.
Season container	Season	Map season title, season number where available, images, offers, and parentExternalId pointing to the show.
Episode	Episode	Map episode title, episode number where available, duration, broadcast date, images, offers, and parentExternalId pointing to the season.
Sports hierarchy entity	Confederation, Competition, Stage, Team, or Persona	Map hierarchy metadata and confirm parent-child rules with the platform team before ingestion.
Linear or live channel	Channel	Map channel metadata, images, offers, and destination fields required by the project.

Step 4B: Map the Common item fields

Most item schemas share these core fields:

Field	Mapper Responsibility
externalId	Generate a stable ID from the source system. Do not use a value that changes between runs.
itemType	Set the exact AXIS item type selected for the source record.
title	Map at least one localized title object with cultureName and text.
offerGroups	Resolve offer group and offer IDs from cached platform reference data.
ratings	Resolve rating id and systemId pairs from cached rating systems.
segmentationTags	Map only platform-defined external IDs.
categories, keywords	Map source taxonomy values where relevant.
availabilityWindows	Map each availability period with key, start, and end.
images	Map externally accessible image URLs with approved imageType and cultureName.
parentExternalId	Set for child records after the parent source record has a stable external ID.
extensions	Map item-type-specific metadata.
customValues	Map integration-specific key/value fields, such as media identifiers where required.
customDestination	Map an external URL or AXIS page destination where the item should link elsewhere.

Example of a minimal schema-backed item payload:

{
  "externalId": "movie-123",
  "itemType": "Movie",
  "title": [
    {
      "cultureName": "en-GB",
      "text": "The Dark Knight"
    }
  ],
  "offerGroups": [
    {
      "id": "offer-group-id",
      "offers": [
        {
          "id": "offer-id"
        }
      ]
    }
  ]
}

Example of a hierarchical child item payload:

{
  "externalId": "episode-s01e03",
  "itemType": "Episode",
  "parentExternalId": "season-s01",
  "title": [
    {
      "cultureName": "en-GB",
      "text": "The Pilot"
    }
  ],
  "offerGroups": [
    {
      "id": "offer-group-id",
      "offers": [
        {
          "id": "offer-id"
        }
      ]
    }
  ]
}

Step 4C: Map type-specific extensions

Use extensions for metadata that belongs to a specific item type, which can be bespoke per project.

Examples from the schema reference include:

Item Type	Extension Examples
Movie	SubType, Genres, Advisory, Duration, ReleaseYear, BroadcastDate, Cast, Crew, Copyright, MediaFiles
Program	SequenceNumber, SubType, Location, Venue, Genres, Advisory, Duration, ReleaseYear, BroadcastDate, EventDate, Cast, Crew, Copyright
Season	SeasonNumber, Genres, Advisory, ReleaseYear, Cast, Crew, ShowTitle, ShowId, SeasonId, VideoId, MediaFiles
Episode	EpisodeNumber, Genres, Advisory, Duration, ReleaseYear, BroadcastDate, ShowId, SeasonId, ShowTitle, SeasonNumber, EpisodeTitle, VideoId, MediaFiles

Do not put type-specific metadata at the top level unless the item schema defines it there. Keep custom and type-specific fields inside extensions, customValues, or the documented schema location.

Step 4D: Map item relations

The schema reference includes payloads for creating and deleting item relations.

A relation payload requires:

• relationType
• relatedItemIds

Example:

{
  "relationType": "related",
  "relatedItemIds": [
    "item-id-1",
    "item-id-2"
  ]
}

The mapper should produce relation payloads only after the related items are known and the relation type has been confirmed from reference data.

The validator should reject relation payloads when:

• relationType is missing
• relationType is not enabled in platform reference data
• relatedItemIds is empty
• a related item ID is unknown
• the payload mixes create and delete intent ambiguously

Step 5: Validate before submit

Validate payloads locally where possible before submitting to AXIS.

Check out this:

• itemType maps to a supported item type for your project
• Rating values resolve to valid id and systemId pairs
• Segmentation tags use known external IDs
• Relation types are enabled before use
• Offer groups reference existing offer group IDs and offer template IDs
• Child items reference parents that already exist in Catalog
• Schedule item externalId values are unique within the schedule
• Schedule startAt and endAt values are ISO 8601 UTC timestamps
• All schedule items fall within the same calendar day
• Custom asset extensions match the requested JSON schema, where applicable

Add schema-specific validation before calling axis-api-ingestion:

Schema Rule	Validator Check
Required fields	Ensure the payload includes the schema-required fields for the selected item type.
Localized text arrays	Ensure each localized text object includes cultureName and text.
itemType value	Ensure the value matches the schema exactly, for example Movie, not movie, unless the platform team confirms otherwise.
Offer groups	Ensure offerGroups contains known group IDs and offer IDs.
Ratings	Ensure every rating object has both id and systemId.
Images	Ensure each image has an accessible url and an approved imageType.
Availability windows	Ensure each window includes a valid key, start, and end.
Parent references	Ensure parent items are ingested before children and that parentExternalId points to the correct parent.
Extensions	Ensure type-specific extension fields match the schema expected for that item type.
Relations	Ensure relation payloads include relationType and relatedItemIds.

Step 6: Submit items

Items are ingested one at a time. The ingestion service uses externalId to determine whether to create or update the item.

POST /v1/{tenantId}/items
Content-Type: application/json
Authorization: Bearer {token}

For hierarchical content, submit parent records before child records:

1. Show
2. Season
3. Episode

Track mappings from your externalId to the platform item ID where needed.

For sports or event hierarchies, confirm the required order for Confederation, Competition, Stage, Team, and Persona with the platform team. Submit parent or container entities before dependent child entities or relations.

Step 7: Submit, replace, or delete schedules

A schedule represents one day of EPG data for a single channel.

Create or submit each channel/day schedule to:

POST /v1/{tenantId}/schedules/{externalId}
Content-Type: application/json
Authorization: Bearer {token}

Use a stable schedule externalId, such as:

bbc-one-2024-10-15

The schedule ID is deterministic, derived from the channel ID and date label, so submitting the same channel and date combination is idempotent.

When updating an existing schedule, use the source-documented PATCH endpoint and send the full replacement set of schedule items. The existing schedule items for that day are replaced entirely.

PATCH /v1/{tenantId}/schedules/{externalId}
Content-Type: application/json
Authorization: Bearer {token}

When deleting a schedule, use:

DELETE /v1/{tenantId}/schedules/{externalId}
Authorization: Bearer {token}

Step 8: Process responses

Status	Meaning	Handling
200 OK	All steps succeeded	Log success and store mappings if needed
206 Partial Content	Some steps failed after partial progress	Read messages or reports, fix the failed step, then retry with forceUpdate=true
400 Bad Request	Workflow failed before meaningful work was done	Check the message, fix the payload, and retry
401 Unauthorized	Token is missing or expired	Refresh the JWT and retry
503 Service Unavailable	Feature flag is disabled	Contact the platform team

Step 9: Verify with reports

Every ingest call can be verified using the Reports API.

GET /v1/{tenantId}/reports
Authorization: Bearer {token}

GET /v1/{tenantId}/reports/{workflowId}
Authorization: Bearer {token}

Use reports to inspect:

• Workflow status
• Workflow type
• Source externalId
• Failed step names
• Error messages
• Workflow creation time

Step 10: Implement, retry and repair

For 206 Partial Content:

1. Inspect the response messages or report steps.
2. Fix the failed step, such as an inaccessible image URL.
3. Resubmit the same payload with forceUpdate=true.

POST /v1/{tenantId}/items?forceUpdate=true
Authorization: Bearer {token}

For stale reference data:

1. Re-fetch the affected reference data from axis-api-pm.
2. Rebuild the payload.
3. Resubmit after correcting the value.

For invalid parent references:

1. Confirm the parent item exists in Catalog.
2. Ingest the parent first if missing.
3. Retry the child item.

For duplicate schedule item IDs:

1. Deduplicate schedule items in your feed.
2. Resubmit the corrected schedule.

For schema validation failures:

1. Identify which schema rule failed.
2. Fix the source mapping or reference data.
3. Rebuild the payload.
4. Resubmit only after local validation passes.

For invalid item relations:

1. Confirm the relation type exists in axis-api-pm.
2. Confirm the related item IDs exist.
3. Rebuild the relation payload.
4. Retry the create or delete relation operation only after the payload is valid.

Step 11: Add reconciliation

For large catalog feeds, run a reconciliation job.

A reconciliation job can compare:

• Source feed records
• Submitted payload checksums
• Recent ingestion reports
• Known partial or failed workflows
• Expected parent-child relationships
• Expected relation to payloads
• Expected item count by itemType

This helps catch records that were skipped, partially ingested, failed during a batch run, or mapped to the wrong schema.

Step 12: Add monitoring and alerts

For batch operations, track enough information to detect failures and partial successes.

Useful signals include:

• Submitted item count
• Successful workflow count
• Partial success count
• 400 validation failure count
• 401 authentication failure count
• 206 image upload failure count
• Failed workflow steps from reports
• Report polling failures
• Reconciliation differences
• Schema validation failures by item type
• Missing parent item failures
• Invalid relation payload failures

Alert when reports show failed workflows, repeated partial successes, or unexpected reconciliation gaps.

Production Checklist

Before go-live, confirm:

• Reference data is fetched from axis-api-pm.
• Reference data cache TTLs match the source recommendations.
• Payloads are validated before submission.
• Item schemas are available to the mapper and validator.
• Required fields are validated for each supported item type.
• Movie, Program, Event, Trailer, Show, Season, Episode, Confederation, Competition, Stage, Team, Persona, and Channel mappings are implemented where used by the project.
• Type-specific extensions are validated before submission.
• Custom extension schemas are requested and applied where needed.
• Parent-child ordering is enforced for hierarchical content.
• Item relation create and delete payloads are validated before submission.
• Schedule create, update, and delete behavior is implemented with the documented endpoints.
• Bulk ingestion concurrency is limited per tenant, starting with 5-10 parallel requests unless the platform team approves more.
• Token refresh is implemented.
• 206 Partial Content recovery is implemented.
• Reports are checked after ingestion.
• Daily schedule syncs poll reports for failures.
• Reconciliation runs for large catalogue feeds.
• Common error handling is documented for operators.