The COVID-19 well being disaster created a dramatic shift in our existence. With work, college, and leisure going down in residence environments now, it’s no shock that display time is up. Nielsen experiences that not solely did total media consumption soar considerably within the first quarter of 2020, the share of video streaming in complete time spent on-line rose from 19% in fourth-quarter 2019 to 25% in second-quarter 2020. By no means has there been extra stress to ship high-quality streaming experiences.
Retaining finish customers engaged is the important thing to profitable streaming. To measure consumer engagement, you want insights into the efficiency of streaming high quality. Visibility into video supply efficiency requires a standard set of metrics for measuring streaming high quality.
To raised perceive the correlation between video high quality and viewer habits, Akamai partnered with biometric analysis agency Sensum to measure the reactions of greater than 1,000 folks to a video viewing expertise. Respondents have been divided into two teams: One seen a video clip delivered at a excessive decision with no buffering, and the opposite noticed the identical phase at a decrease decision with a buffering occasion throughout a key scene. Reactions have been measured utilizing instruments corresponding to galvanic pores and skin response displays and facial coding software program. The outcomes:
Greater-quality streams generated 19.8% extra emotional engagement
Rebuffering induced a 16% enhance in destructive feelings, 9% enhance in disgust, 7% enhance in unhappiness, and eight% lower in focus, based on facial coding strategies
Greater than three-quarters of contributors stated they’d cease utilizing a service if an issue like rebuffering occurred a number of instances
Streaming suppliers use a wide range of instruments to measure viewing expertise high quality. Two widespread units of metrics are High quality of Expertise (QoE) and High quality of Service (QoS). Whereas these are helpful measurements, they’ve some blind spots in the case of measuring video efficiency.
QoE determines the viewer expertise and notion of the service high quality measured on the shopper machine. QoS offers server-side knowledge to assist pinpoint the supply of enchancment or degradation in efficiency throughout the infrastructure, corresponding to a content material supply community (CDN). QoS is usually characterised as offering operational metrics — transactional knowledge primarily used for real-time operational choices — and may be tough to narrate to consumer expertise. Neither measurement offers a full image of video high quality and efficiency.
To make sure unified measurement of consumer expertise throughout enterprise and operations groups, Akamai makes use of High quality of Efficiency (QoP) metrics. QoP metrics are parameters that outline operational knowledge factors to enhance system efficiencies and supply visibility for enterprise choices by serving as proxy QoE metrics. These measurements present extra visibility into the service high quality of the CDN and carefully relate to QoE.
An instance of a QoP metric is an adaptive bitrate streaming state of affairs. When you’ve got video segments which can be six seconds lengthy, and the CDN takes greater than six seconds to ship that phase, there will likely be rebuffering on the shopper participant. QoP measures the CDN service high quality habits and serves as a proxy metric to client-side rebuffering. With this perception, you possibly can outline efficient QoP benchmarks to your video expertise service supplier that present visibility into end-user experiences.
Study extra about how QoP metrics measure on-line video high quality.
With massive gatherings on maintain, streaming occasions are a well-liked various to in-person experiences like concert events, theater performances, conventions, and sporting matches. Let’s look at a real-world state of affairs the place video high quality metrics are crucial: a large stay occasion.
Delivering a profitable streaming occasion on a worldwide degree is constructed upon assessments and processes which can be outlined months — or in some instances, years — upfront. How do you make sure that your group can ship? Dwell-streaming success depends closely on the CDN platform capability, streaming structure, and deployment experience, in addition to correct planning preparation and assist.
Stepping into planning mode early, armed with historic knowledge and projections, lets you perceive the capability wanted for the estimated viewers dimension, structure optimizations, and assist required for the dimensions of the occasion. And Akamai companions with our prospects at every step of this stage to assist guarantee seamless execution.
Correct planning contains an end-to-end architectural evaluation to make sure every element of the streaming workflow — from origin to the final mile — is resilient and performs successfully at scale. For instance, establishing a method to form last-mile bandwidth pipe dimension to accommodate COVID-era bandwidth constraints helps content material suppliers and community house owners ship predictable efficiency.
Preparation contains making inside and exterior preparations for a profitable streaming occasion. It begins with dry runs to align runbooks internally with assist, product, platform, and govt groups. Externally, buyer communications, fallbacks, and contingency plans are readied to deal with any sudden points. Measurement instruments, metrics, and thresholds are clearly outlined at this stage to correctly assess the efficiency of the occasion.
When it’s time to broadcast, streaming high quality is reviewed each day. Every occasion is carefully watched as carried out, from begin to end. In response to efficiency reporting, the actions that have been predetermined within the preparation stage are rolled out to repair or optimize experiences.
Thoughtfully assessing the efficiency of the occasion earlier than shifting on is essential for steady enchancment. How did the occasion carry out relative to the goal metrics? Do any infrastructure points must be addressed? It’s essential to work with the entire stakeholders to collect the learnings and apply them to the following occasion.
One of many largest streaming platforms in Asia partnered with Akamai to stay stream one of many greatest sporting leagues on the earth the place the viewership of the occasion on cell units has been rising exponentially. Akamai has been working with this buyer because the launch of their OTT platform to offer scale and low latency so viewers can take pleasure in a broadcast-quality expertise on cell units.
To organize for the event, Akamai helped the client outline the suitable efficiency metrics — enterprise priorities, out there knowledge units, monitoring telemetry, toolsets, efficiency thresholds, and runbook performs (mitigation, analysis, elevation, and escalation) ought to an sudden problem must be addressed.
We additionally measured the QoP of the Akamai platform based mostly on throughput, reliability, and server-side rebuffering estimation, a real indicator of CDN efficiency that carefully correlates with QoS and QoE metrics. And at last, there was server-side QoS monitoring.
An essential metric given distinctive significance throughout this occasion was client-side errors; this was crucial because the majority of the viewers (greater than 80%) could be watching the video games from their cellphones, the place the impression of consumer errors is magnified. If a problem arose, we needed to be forward of it and mitigate it as quick as potential earlier than customers began shifting away.
And we noticed that unfold throughout one of many crucial video games, the place there have been widespread 403s reported by a lot of customers. These really impacted the client’s assist desk operations as nicely, with the variety of tickets skyrocketing. With proactive monitoring, we have been capable of determine the foundation trigger to be ISP-induced header insertion and shortly diverted site visitors away from the community areas the place this was prevalent. In the long run, we labored carefully with the ISPs and the client to resolve the problem completely.
Along with proactive monitoring and mitigation processes, Akamai deployed instruments to observe efficiency, created alerting techniques, and developed communications channels (each tactical and elevated), in addition to skilled and practiced for the occasion. When thresholds have been shut, we checked out QoP key efficiency indicator (KPI) knowledge to get rid of any issues about CDN efficiency from the Akamai platform — this allowed us to have interaction immediately with the ISP extra shortly, saving loads of time.
The ultimate sport of the occasion dealt with greater than 18 million concurrent streaming viewers, making it one of many largest streaming occasions by viewers dimension. Setting these information whereas ensuring that viewers are having fun with a stay sporting occasion will all the time carry the problem of expertise assist and supply functionality. Akamai has continued to ship by managing geographical spikes in site visitors yearly, whereas matching streaming high quality expectations.
*** This can be a Safety Bloggers Community syndicated weblog from The Akamai Weblog authored by Harish Menon. Learn the unique put up at: http://feedproxy.google.com/~r/TheAkamaiBlog/~3/2I7XqXUTK1k/metrics-that-matter-measuring-streaming-performance.html
direct marketing spotify,kpi music industry,metrics that matter book,spotify engagement metrics,music metric,it metrics that matter,netflix key performance indicators,kpi business insider,netflix efficiency score,netflix starters completers,how does netflix measure customer experience,how does netflix measure employee performance,video streaming performance metrics,facebook live stream metrics,key metrics for netflix,how to measure success of live stream,vimeo livestream metrics,music streaming metrics,measuring metrics pdf,metrics that matter pdf,metrics that matter app,the measurement advantage bain,how to measure consulting success,kpi for service company,how does netflix measure performance,netflix performance metrics,real time video performance is measured by what metric,streaming metrics,kpi for video streaming