Video segments and manifests

Streaming does not send one large file. The video is cut into short segments, usually a few seconds each, and a manifest lists them with their URLs. The player downloads the manifest first, then fetches segments one after another from the nearest cache.

The big idea

Think of a long train journey split into numbered short tickets. A conductor's list says where each ticket is kept. The traveler can pick up the next ticket from a different place without changing the whole journey.

The picture breaks at the seams. Segments of different qualities must cut the video at the same moments, or the player cannot switch cleanly between them. That is why encoders force keyframes at shared timestamps.

Walking through it

A 10-minute video cut into 4-second segments gives 600 ÷ 4 = 150 segments per rendition. Five renditions give 150 × 5 = 750 segment files in total.

The manifest for HLS is a plain text file. The master manifest lists the renditions and their bandwidth:

#EXTM3U
#EXT-X-STREAM-INF:BANDWIDTH=2500000,RESOLUTION=1280x720
720p/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=640x360
360p/index.m3u8

Each rendition has its own playlist of segments:

#EXTINF:4.0,
seg-0001.m4s
#EXTINF:4.0,
seg-0002.m4s

The player can see from the master manifest that 720p needs about 2.5 Mbps and 360p about 800 kbps, and it can choose without downloading anything else. DASH uses a different manifest format, and CMAF lets HLS and DASH share the same segment files.

Segment length is a trade-off. Longer segments code more efficiently but switch quality more slowly. Shorter segments switch faster but add overhead. Two to four seconds is a common choice for on-demand video.

Common mistakes

• Misaligned keyframes. If renditions do not cut at the same moments, a quality switch shows a visible glitch.
• Very long segments. The player cannot react to a bandwidth drop until the current segment finishes.
• Serving segments from origin for every view. Segments are immutable and should be cached at the edge.
• Unsigned manifests. A leaked manifest can be shared with anyone who has the link.

Check yourself

1. How many 4-second segments does a 10-minute video have per rendition?

Answer: 150, since 600 seconds ÷ 4 seconds = 150.

2. Why must renditions cut at the same moments?

Answer: So the player can switch from one quality to another at a segment boundary without a glitch.

Key takeaways

• A manifest lists the renditions and the segment URLs, and the segments are short files.
• Segments are immutable, so they cache well on a CDN.
• Align keyframes across renditions, so switching stays seamless.
• HLS and DASH differ in manifest format. CMAF lets them share segments.

Further learning

Next, see how those renditions are produced in transcoding as a DAG.