Skip to content
Capacity and Reliability

Capacity and Reliability

Use System → Capacity & Reliability or the web CLI to test real playback paths under controlled load.

!!! warning “Use an Isolated Test Environment” Do not load-test production customers or channels. Use a test license, dedicated User, continuous low-bitrate source, and a maintenance window.

Run an Assessment

!!! info “Access and Licensing” Only Full Administrators can open this page. It requires the dedicated capacity_assessment feature and does not bypass User or license limits.

  1. Select one or more online target servers with an HTTP listener.
  2. Select a Stream, Movie, or Episode present in the test User’s playlist.
  3. Select the User. Enter a password only for password authentication; IP authentication uses the load generator’s real source IP.
  4. Select HLS, HTTP-TS, or both. Movie and Episode tests use HLS; HTTP-TS is available for Streams. RTMP is not tested.
  5. Set the total player count and a duration from 10 to 3600 seconds. The total is divided across selected protocols.
  6. Start the test and monitor CPU, memory, network, progress, errors, bandwidth, and P99 latency.
  7. Export the result only after the task reaches a final state.

!!! important “The Test Uses Production Behavior” Every player requests a real playlist and follows normal authentication, SID, cache, CDN, delivery, and license admission paths. Existing caches remain active, so database load reflects production caching.

Read the Result

OutcomeMeaning
StableConnected, stayed active until completion, had no failed requests, and did not rebuffer
FreezeAn HLS player rebuffered at least once
Other issuesConnection failed, ended early, or had another error without an HLS rebuffer
IncompleteThe assessment stopped before every player finished

Use these acceptance indicators for a connection test:

  • connection success and retention at least 99.9%;
  • request error rate no higher than 0.1%;
  • no sustained goroutine, RSS, cache, or database-connection leak over 30 minutes;
  • active state returning to baseline within the normal cleanup window.

??? info “How Buffering Is Classified” HLS models an initial three-segment buffer. A download taking at least 80% of its EXTINF duration is slow; buffer exhaustion longer than 250 ms is a rebuffer. HTTP-TS marks a data gap longer than one second as a potential rebuffer, but reports unstable players under Other issues because it has no equivalent HLS Freeze count.

!!! important “Task Status Is Not Playback Quality” Completed means the task ran, not that playback passed. JSON passed and the CLI exit code remain strict: every player must be stable and request errors must be zero.

Test Boundaries

  • One UI run supports up to 100,000 logical players. This is an operational guardrail, not extra licensed capacity.
  • User Max Connections and license Max Concurrent Connections / Max Delivery Bandwidth remain enforced.
  • Every target must be inside the selected User’s allowed playback/CDN scope.
  • Passwords stay in memory and are not written to task state, reports, or logs.
  • /server/stats charts sample every three seconds while the page is open; browser samples are not restored after refresh.
  • The load generator runs in the main server process, so its CPU, memory, and network include client-generation overhead.
  • One source IP may exhaust ephemeral ports or file descriptors before Engine. Large tests should use multiple source addresses or independent generators.

Run one explicit player count at a time. Increase it between runs and allow resources to recover; automatic multi-stage probing can hide accumulated memory or unrecovered state.

Delivery Bandwidth Targets

average_mbps counts received media body bytes over the full test duration. Playlist and HLS manifest bytes are excluded.

PackageSustained target
Base1,000 Mbps
Standard5,000 Mbps
Professional10,000 Mbps
Enterprise25,000 Mbps

Sustain each target for at least 10 minutes and validate HLS, HTTP-TS, TLS, and CDN Relay separately.

!!! note A 25 Gbps result requires suitable 25/40/100 GbE hardware. Total NIC traffic is not a substitute for delivered media bytes.

Run From the CLI

The CLI retains --benchmark-* options for existing automation:

/opt/xaccel-engine/web \
  --benchmark \
  --benchmark-playlist 'https://engine.test/stream/playlist' \
  --benchmark-auth password \
  --benchmark-username load-test \
  --benchmark-password-file /run/secrets/xae-benchmark-password \
  --benchmark-protocol http-ts \
  --benchmark-players 25000 \
  --benchmark-duration 30m \
  --benchmark-output connections.json

Use --benchmark-auth ip without username/password for IP authentication, --benchmark-protocol hls for HLS, and repeat --benchmark-local-address IP to distribute source addresses. The JSON report excludes URLs and credentials; an output file uses 0600 permissions.

Certify Capacity

  1. Record hardware, kernel, Engine, Go, FFmpeg, TLS, and network topology.
  2. Confirm the test license, User limits, media source, and CDN pool.
  3. Run a five-minute smoke test at 10% of the target.
  4. Run a separate 30-minute connection stability test.
  5. Wait for connections, caches, and system resources to return to baseline.
  6. Run a separate 10-minute sustained bandwidth test.
  7. Repeat for each protocol, TLS mode, CDN path, and server combination.
  8. Save JSON, charts, metrics, and sanitized logs.
  9. Rule out generator, source, and network bottlenecks.
  10. Certify a package figure only after the reference hardware passes every scenario.

!!! important “Publish the Test Conditions” A passing run is not a guarantee for every server. State the reference hardware, protocol, bitrate, TLS use, connection count, duration, software version, and measurement tolerance with every published result.