Advanced Assertions for Video Streaming Tests with ULPVideoAnalysisUtils

When running performance tests on video streaming platforms, you often need to go beyond basic pass/fail assertions. Whether you are testing HLS, MPEG-DASH, or Smooth Streaming services, having granular control over your test validations is essential for meaningful results.

We are excited to introduce the ULPVideoAnalysisUtils utility class, designed to empower testers with powerful tools for creating custom assertions in their video streaming performance tests.

Why ULPVideoAnalysisUtils ?

The UbikLoadPack Video Streaming Plugin generates different types of sample results during test execution :

• Manifest requests: Downloads of playlist/manifest files (.m3u8, .mpd)
• Chunk requests: Downloads of video/audio segments
• VideoStreamingSampleResult: Aggregated metrics for the entire streaming session

Each type carries specific information, and accessing this data programmatically allows you to create precise, targeted assertions. That’s where ULPVideoAnalysisUtils comes in.

Key Features

Identify Request Types

Easily distinguish between different request types in your assertions :

import com.ubikingenierie.jmeter.plugin.video.client.ULPVideoAnalysisUtils

// Check if this is a manifest request
if (ULPVideoAnalysisUtils.isManifestRequest(prev)) {
    log.info("Processing manifest request: " + prev.getURL())
}

// Check if this is a chunk request
if (ULPVideoAnalysisUtils.isChunkRequest(prev)) {
    log.info("Processing chunk request: " + prev.getURL())
}

// Check if this is a video streaming aggregated result
if (ULPVideoAnalysisUtils.isVideoStreamingSampleResult(prev)) {
    log.info("Processing video streaming sample result")
}

This is particularly useful when you want to apply different validation rules based on the request type.

Extract Response Headers

Retrieve specific header values from responses :

import com.ubikingenierie.jmeter.plugin.video.client.ULPVideoAnalysisUtils

// Get the Content-Type header
String contentType = ULPVideoAnalysisUtils.retrieveResponseHeaderValue(prev, "Content-Type")

// Validate CDN cache status
String cacheStatus = ULPVideoAnalysisUtils.retrieveResponseHeaderValue(prev, "X-Cache")
if (cacheStatus != null && cacheStatus.contains("HIT")) {
    log.info("Request served from CDN cache")
}

// Check for specific CDN headers
String cdnNode = ULPVideoAnalysisUtils.retrieveResponseHeaderValue(prev, "X-CDN-Node")

The header name matching is case-insensitive, so "content-type" and "Content-Type" will work the same way.

Access Quality of Experience (QoE) Metrics

For video streaming tests, QoE metrics are crucial. Access them directly :

import com.ubikingenierie.jmeter.plugin.video.client.ULPVideoAnalysisUtils

// Get the lag ratio (percentage of time spent buffering vs playing)
float lagRatio = ULPVideoAnalysisUtils.getLagRatio(prev)
if (lagRatio > 5.0) {
    AssertionResult.setFailure(true)
    ULPVideoAnalysisUtils.overrideResponseCodeAndMessage(prev, "LAG_TOO_HIGH", "Lag ratio too high: " + lagRatio + "% (threshold: 5%)")
}

// Get lag ratio excluding initial buffer fill
float lagRatioWithoutBuffer = ULPVideoAnalysisUtils.getLagRatioWithoutBufferFill(prev)

// Get the buffer fill duration in milliseconds
long bufferFillMs = ULPVideoAnalysisUtils.getBufferFillDurationInMs(prev)
if (bufferFillMs > 3000) {
    log.warn("Initial buffer fill took more than 3 seconds: " + bufferFillMs + "ms")
}

Override Response Codes

Customize response codes based on your business logic.

Why use overrideResponseCodeAndMessage instead of AssertionResult.setFailureMessage?

When using JMeter’s AssertionResult.setFailure(true) and AssertionResult.setFailureMessage(...), the failure information is only stored in the assertion result and may not be visible in :

• The JTL/CSV results file
• The HTML Dashboard Report
• Response code statistics and graphs

By using ULPVideoAnalysisUtils.overrideResponseCodeAndMessage(...), you directly modify the sample’s response code and message. This ensures that :

• Your custom error codes appear in the Response Codes Per Second graph
• Error messages are visible in the CSV/JTL output
• You can filter and analyze results by custom response codes in the HTML report
• The APDEX calculation takes your custom codes into account

This approach gives you full visibility into your test results and makes analysis much easier.

You must call AssertionResult.setFailure(true) before overrideResponseCodeAndMessage to flag assertion as failed.

import com.ubikingenierie.jmeter.plugin.video.client.ULPVideoAnalysisUtils

// Mark as failed if lag ratio exceeds threshold
float lagRatio = ULPVideoAnalysisUtils.getLagRatio(prev)
if (lagRatio > 10.0) {
    AssertionResult.setFailure(true)
    ULPVideoAnalysisUtils.overrideResponseCodeAndMessage(prev, "LAG_ERROR", "Excessive buffering detected: " + lagRatio + "%")
}

// Custom success code for specific scenarios
if (ULPVideoAnalysisUtils.isVideoStreamingSampleResult(prev) && lagRatio < 1.0) {
    AssertionResult.setFailure(true)
    ULPVideoAnalysisUtils.overrideResponseCodeAndMessage(prev, "EXCELLENT_QOE", "Smooth playback achieved")
}

Practical Examples

Example 1 : Validate CDN Performance

Create a JSR223 Assertion to verify CDN cache behavior :

import com.ubikingenierie.jmeter.plugin.video.client.ULPVideoAnalysisUtils

// Only check chunk requests (not manifests)
if (ULPVideoAnalysisUtils.isChunkRequest(prev)) {
    String cacheStatus = ULPVideoAnalysisUtils.retrieveResponseHeaderValue(prev, "X-Cache-Status")
    
    if (cacheStatus == null) {
        // No cache header found - might indicate CDN misconfiguration
        log.warn("No X-Cache-Status header found for chunk: " + prev.getURL())
    } else if (cacheStatus.equalsIgnoreCase("MISS")) {
        // Log cache misses for analysis
        log.info("Cache MISS for: " + prev.getURL())
    }
}

Example 2 : QoE-Based Pass/Fail Criteria

Implement business-driven success criteria :

import com.ubikingenierie.jmeter.plugin.video.client.ULPVideoAnalysisUtils

// Only validate on the aggregated video streaming result
if (ULPVideoAnalysisUtils.isVideoStreamingSampleResult(prev)) {
    
    // Define QoE thresholds
    float maxLagRatio = 5.0           // Maximum 5% lag
    long maxBufferFillMs = 4000       // Maximum 4 seconds to start playback
    
    float lagRatio = ULPVideoAnalysisUtils.getLagRatio(prev)
    long bufferFillMs = ULPVideoAnalysisUtils.getBufferFillDurationInMs(prev)
    
    StringBuilder errors = new StringBuilder()
    
    if (lagRatio > maxLagRatio) {
        errors.append("Lag ratio ").append(lagRatio).append("% exceeds threshold ").append(maxLagRatio).append("%. ")
    }
    
    if (bufferFillMs > maxBufferFillMs) {
        errors.append("Buffer fill ").append(bufferFillMs).append("ms exceeds threshold ").append(maxBufferFillMs).append("ms. ")
    }
    
    if (errors.length() > 0) {
        AssertionResult.setFailure(true)
        ULPVideoAnalysisUtils.overrideResponseCodeAndMessage(prev, "QOE_THRESHOLD_EXCEEDED", errors.toString())
    }
}

Example 3 : Differentiated Validation by Request Type

Apply different rules for manifests vs chunks :

import com.ubikingenierie.jmeter.plugin.video.client.ULPVideoAnalysisUtils

if (ULPVideoAnalysisUtils.isManifestRequest(prev)) {
    // Manifests should be served quickly
    if (prev.getTime() > 500) {
        AssertionResult.setFailure(true)
        ULPVideoAnalysisUtils.overrideResponseCodeAndMessage(prev, "MANIFEST_TOO_SLOW", "Manifest response time too slow: " + prev.getTime() + "ms")
    }
    
    // Check manifest content type
    String contentType = ULPVideoAnalysisUtils.retrieveResponseHeaderValue(prev, "Content-Type")
    if (contentType != null && !contentType.contains("mpegurl") && !contentType.contains("dash+xml")) {
        log.warn("Unexpected manifest content type: " + contentType)
    }
    
} else if (ULPVideoAnalysisUtils.isChunkRequest(prev)) {
    // Chunks can take longer but should be consistent
    if (prev.getTime() > 2000) {
        log.warn("Slow chunk download: " + prev.getTime() + "ms for " + prev.getURL())
    }
}

API Reference

Method	Description	Return Type
isManifestRequest(SampleResult)	Checks if the sample is a manifest request	boolean
isChunkRequest(SampleResult)	Checks if the sample is a chunk request	boolean
isVideoStreamingSampleResult(SampleResult)	Checks if the sample is an aggregated video streaming result	boolean
retrieveResponseHeaderValue(SampleResult, String)	Retrieves a specific header value (case-insensitive)	String or null
getLagRatio(SampleResult)	Gets the lag ratio percentage	float
getLagRatioWithoutBufferFill(SampleResult)	Gets the lag ratio excluding initial buffer fill	float
getBufferFillDurationInMs(SampleResult)	Gets the initial buffer fill duration in milliseconds	long
overrideResponseCodeAndMessage(SampleResult, String, String)	Overrides the response code and message	void

Getting Started

To use ULPVideoAnalysisUtils in your JMeter test plan :

1. Add a JSR223 Assertion or JSR223 PostProcessor to your Video Streaming Sampler
2. Select Groovy as the scripting language
3. Import the utility class: com.ubikingenierie.jmeter.plugin.video.client.ULPVideoAnalysisUtils
4. Use the prev variable to access the current sample result

Conclusion

The ULPVideoAnalysisUtils class provides a clean, type-safe API for working with video streaming sample results in JMeter. By leveraging these utilities in your JSR223 assertions, you can :

• Create precise, targeted validations
• Implement QoE-based success criteria
• Gain deeper insights into CDN and streaming performance
• Build more meaningful and actionable test reports

Start using ULPVideoAnalysisUtils today to take your video streaming performance tests to the next level !

You’ll probably also like :

• Performance Testing Low Latency Dash servers with UbikLoadPack
• Performance Testing Low Latency HLS servers
• Load testing MPEG-DASH Video Streaming with Apache JMeter and UbikLoadPack
• Video Streaming: CMAF and Low-Latency
• Cases studies