What Is SSIM? Structural Similarity Index Explained for Video Quality

Two encodes of the same clip come back from your pipeline. One scores 0.98 SSIM and looks slightly mushy in the grass. The other scores 0.88 and looks fine to everyone on the team.

That happens more often than encoding tutorials admit.

SSIM was built to fix the biggest flaw in PSNR: pixel-difference math has almost nothing to do with what human eyes notice. It mostly succeeds. But it succeeds within limits you need to know before you gate a release on it.

What Is SSIM?

SSIM (Structural Similarity Index Measure) is a full-reference image and video quality metric that predicts perceived quality by comparing the luminance, contrast, and structure of a processed frame against an original reference frame. It outputs a single number between −1 and 1, where 1 means the two frames are identical.

“Full-reference” is the important qualifier. SSIM can’t look at a video and tell you whether it’s good. It needs the pristine source alongside the output, and it reports how much the second one drifted from the first.

So it’s a tool for grading your own video encoding, not for grading someone else’s stream.

Human vision is tuned to structure. That’s the idea behind SSIM, published by Zhou Wang, Alan Bovik, Hamid Sheikh, and Eero Simoncelli in April 2004.

We notice when edges blur, textures flatten, or gradients band. We barely notice a uniform brightness shift across the whole frame. So instead of summing squared pixel errors, SSIM breaks the comparison into three perceptual pieces and multiplies them together.

That paper has been cited more than 50,000 times, won the IEEE Signal Processing Society Best Paper Award in 2009, and earned its authors a Primetime Engineering Emmy in 2015. You can read the original SSIM research if you want the full derivation.

Searched for SSIM and wanted airline scheduling? You want IATA’s Standard Schedules Information Manual, which shares the acronym and nothing else. Everything below is the video quality metric.

Two other metrics show up in the same conversations. PSNR is the older pixel-error metric SSIM replaced, and VMAF is Netflix’s machine-learning metric that fuses several models and adds motion. Both get compared against SSIM below.

The SSIM Formula Explained

SSIM doesn’t run on whole frames. It slides a small window across the image, scores each window, and averages the results.

The full-frame number you see reported is technically MSSIM: mean SSIM across every window position.

For a single pair of windows x and y, the combined SSIM formula is:

                (2·μx·μy + c₁)(2·σxy + c₂)
SSIM(x,y) = ───────────────────────────────────
             (μx² + μy² + c₁)(σx² + σy² + c₂)

That compact form is three separate comparisons multiplied together. Splitting them apart makes the metric readable.

Luminance comparison

l(x,y) = (2·μx·μy + c₁) / (μx² + μy² + c₁)

This compares mean brightness. μx and μy are the average pixel values in each window.

If both windows average the same brightness, this term goes to 1. A gamma shift or exposure change moves it down.

Contrast comparison

c(x,y) = (2·σx·σy + c₂) / (σx² + σy² + c₂)

This compares standard deviation, or how much variation exists inside the window.

Heavy compression flattens texture, which drops σ in the encoded window and pulls this term down. It’s the term that catches detail loss in grass, hair, water, and film grain.

Structure comparison

s(x,y) = (σxy + c₃) / (σx·σy + c₃)

This is the correlation between the two windows after brightness and contrast are normalized out. It asks whether the pattern is still the same pattern.

Blocking artifacts, ringing, and mosquito noise hit this term hardest. It’s also the piece that gives SSIM its name.

The constants and the window

The constants keep the math stable when denominators approach zero:

• c₁ = (k₁·L)² with k₁ = 0.01
• c₂ = (k₂·L)² with k₂ = 0.03
• c₃ = c₂ / 2

L is the dynamic range of the pixel values: 255 for 8-bit content, 1023 for 10-bit. Those k values are the defaults from the 2004 paper, and almost nothing changes them. That’s a large part of why SSIM scores are comparable across tools.

The standard window is an 11×11 circular-symmetric Gaussian with σ = 1.5, weighted so the center pixel counts most.

Some implementations use a flat 8×8 box instead, which is faster and scores slightly differently. Worth knowing if two tools disagree on the same file.

SSIM is normally computed on the luma plane only, since that’s where structure lives. FFmpeg reports Y, U, and V separately plus a combined figure, so you can see whether a problem is luma detail or chroma.

If you’re chasing a chroma-specific artifact, check which video codec and subsampling mode the encode used before you blame the metric.

What Is a Good SSIM Score?

The SSIM value range runs 0 to 1 for any realistic encode, and the useful part of that range is squeezed into the top few percent.

A visibly broken encode still scores around 0.90. That’s the most surprising thing about reading SSIM scores for the first time.

Jan Ozer mapped SSIM scores to subjective ratings and put the practical line around 0.95, roughly equivalent to a VMAF score of 93. Below 0.99 you’re in “perceptible but not annoying” territory. Above it, viewers can’t tell.

Most streaming teams settle on tiers like this:

• 0.95+ for premium and top-of-ladder renditions
• 0.90+ for mid-ladder rungs
• 0.85+ as the floor below which a rung isn’t worth shipping

Now the caveat that matters more than the table. Those numbers are content-dependent, not universal.

Practitioners regularly see clips at 0.88 that nobody complains about and clips at 0.98 that look wrong. Flat animation and screen recordings score high easily because there’s little texture to lose.

Film grain, confetti, water, and crowd shots score low even when they look great, because SSIM reads randomized high-frequency detail as structural change.

So calibrate against your own library instead of importing someone else’s threshold. Encode ten clips that represent your real content, watch them, and find where your eyes say “not good enough.” That number becomes your gate.

Feeding those thresholds back into your bitrate targets is how SSIM turns into savings instead of a dashboard number.

SSIM in dB

FFmpeg also prints a decibel figure next to each SSIM value, computed as −10 · log₁₀(1 − SSIM). It stretches the crowded top of the scale into something easier to read: 0.99 becomes 20 dB, 0.999 becomes 30 dB.

When you’re comparing encodes that all land above 0.98, the dB column shows differences the raw score buries.

Neither number tells you anything about buffering, startup delay, or rebuffer ratio. SSIM grades the pixels. Quality of experience covers everything else, and a stream can score 0.99 while viewers abandon it.

SSIM vs PSNR vs VMAF

The PSNR vs SSIM question comes up constantly, and adding VMAF makes it a three-way decision. Each metric trades accuracy against compute cost differently.

SSIM vs PSNR

PSNR sums squared pixel differences and converts to decibels. It’s nearly free to compute and completely blind to perception.

Shift every pixel by two levels and PSNR tanks while the image looks identical. Blur the texture out of a face and PSNR barely moves.

SSIM fixes exactly that. It normalizes brightness and contrast out of the structure term, so it reacts to the artifacts viewers notice and ignores the ones they don’t. On standard image quality datasets SSIM correlates with human scores around 0.72 against PSNR’s 0.5 to 0.6.

Use PSNR when you want a fast regression signal that a build didn’t break. Use SSIM when you want a number that tracks how the encode looks.

SSIM vs VMAF

VMAF wins on accuracy. It combines Visual Information Fidelity, a Detail Loss Metric, and a motion feature, then trains a regressor on human ratings.

Unlike SSIM, it sees temporal behavior. Its linear 0-to-100 scale also gives you meaningful spread instead of everything crowding above 0.95.

The cost is compute. VMAF runs several times slower than SSIM, and the VIF feature is the expensive part. On a large transcoding catalog that difference is real money and real wall-clock time.

The common split: SSIM as the cheap gate that runs on everything, VMAF as the deeper check on samples, new codecs, and anything shipping to a premium tier.

Types of SSIM: MS-SSIM, DSSIM, and Other Variants

Twenty years of research produced a family of SSIM variants. A few matter in practice.

MS-SSIM (Multi-Scale SSIM)

MS-SSIM computes SSIM at multiple resolutions, typically five scales, halving the image each time, then combines them with fixed weights. Viewing distance and display size change which details are visible, so evaluating across scales tracks perception better than a single pass.

It’s the variant most streaming engineers should default to. Modern encoders reference it directly: x265 exposes ssim-rd for SSIM-aware rate-distortion decisions, and both x264 and x265 support --tune ssim.

DSSIM (Structural Dissimilarity)

DSSIM inverts the scale as (1 − SSIM) / 2, so 0 means identical and higher means worse. It’s convenient when you want an error term that behaves like a loss rather than a score.

SSIM as a loss function

In machine learning, SSIM loss (usually 1 − SSIM or DSSIM) trains super-resolution, denoising, and compression networks.

Models trained on SSIM loss produce sharper output than models trained on mean squared error, which tends toward blurry averages. That’s why SSIM shows up in PyTorch and TensorFlow training loops far from any streaming pipeline.

3-SSIM and 4-SSIM

These weight regions differently. Edges, textures, and smooth areas contribute unequally, since compression artifacts bother viewers more in some regions than others.

CW-SSIM (Complex Wavelet SSIM)

CW-SSIM operates in the complex wavelet domain, which makes it tolerant of small translations, rotations, and scaling. Useful when frames aren’t pixel-aligned, something standard SSIM handles badly.

SSIMPLUS

A commercial metric from SSIMWAVE built on SSIM foundations. It reports on a 0-to-100 scale and adjusts for display size and viewing conditions.

SSIMULACRA2

A modern variant working in the XYB color space, developed as part of the JPEG XL reference tooling. It targets still images, so it shows up in codec comparison work rather than video ladders.

How to Calculate SSIM

Three practical routes: FFmpeg for video, Python for images and experiments, MATLAB if you’re checking against reference implementations.

Calculate SSIM with FFmpeg

The FFmpeg ssim filter is the fastest path. Pass the distorted file first, the reference second:

ffmpeg -i encoded.mp4 -i reference.mp4 \
  -lavfi ssim=stats_file=ssim.log \
  -f null -

The summary line looks like this:

SSIM Y:0.926845 (11.357537) U:0.876798 (9.093807) V:0.860658 (8.559193) All:0.907472 (10.337287)

Each plane gets a raw score with the dB figure in parentheses. Y is luma, the one to watch. All is the weighted combination.

The stats file gives you per-frame data:

n:1 Y:0.931245 U:0.881003 V:0.864221 All:0.912011 (10.556)
n:2 Y:0.929887 U:0.879654 V:0.862990 All:0.910442 (10.480)

Per-frame output is where SSIM earns its keep. A frame-by-frame log exposes the scene that fell apart, which an averaged score hides completely.

Two requirements catch people. Both inputs need matching resolution and frame count.

If your rendition is smaller than the source, upscale it in the filter graph first:

ffmpeg -i encoded_720p.mp4 -i reference_1080p.mp4 \
  -lavfi "[0:v]scale=1920:1080:flags=bicubic[main];[main][1:v]ssim=stats_file=ssim.log" \
  -f null -

Scale the encode up to the reference, not the reference down. Downscaling the source discards the detail you’re trying to measure.

To sweep a CRF range and see where quality stops improving:

for crf in 18 21 24 27 30 33; do
  ffmpeg -y -i source.mp4 -c:v libx264 -crf $crf -preset medium out_$crf.mp4
  echo -n "CRF $crf: "
  ffmpeg -i out_$crf.mp4 -i source.mp4 -lavfi ssim -f null - 2>&1 \
    | grep -o 'All:[0-9.]*'
done

Plot the results and you get a rate-distortion curve for your content. The knee, where more bits stop buying meaningful SSIM, is where your ladder rung belongs.

Run the same sweep per codec when you’re evaluating AV1 encoding or working through an HEVC vs H.264 decision, since the curves sit in different places.

Calculate SSIM in Python

For images and quick experiments, scikit-image ships a reference implementation:

from skimage.metrics import structural_similarity as ssim
import cv2

ref = cv2.imread("reference.png", cv2.IMREAD_GRAYSCALE)
enc = cv2.imread("encoded.png", cv2.IMREAD_GRAYSCALE)

score, diff_map = ssim(ref, enc, full=True)
print(f"SSIM: {score:.4f}")

Setting full=True returns a spatial SSIM map alongside the score. Rendering that map as a heatmap shows you where quality dropped, usually the most useful diagnostic SSIM offers and something the scalar score can’t give you.

For frames as tensors during training, pytorch-msssim provides differentiable SSIM and MS-SSIM implementations that plug straight into a loss function.

Calculate SSIM in MATLAB

MATLAB’s Image Processing Toolbox has SSIM built in, and it’s the implementation many papers check against:

ref = imread('reference.png');
enc = imread('encoded.png');
[score, ssim_map] = ssim(enc, ref);
fprintf('SSIM: %.4f\n', score);

Wrapper tools

ffmpeg-quality-metrics wraps FFmpeg and returns SSIM, PSNR, VMAF, and VIF together as JSON:

pip install ffmpeg-quality-metrics
ffmpeg-quality-metrics encoded.mp4 reference.mp4 --metrics ssim psnr vmaf

Structured output makes this the easiest option for CI pipelines, since you can assert on a field instead of parsing console text.

Advantages of SSIM

Better perceptual correlation than PSNR

The core reason SSIM exists. By separating structure from brightness and contrast, it reacts to blur, blocking, and banding, the artifacts viewers actually report, instead of to pixel arithmetic.

Cheap enough to run on everything

SSIM costs a small fraction of VMAF. You can score an entire catalog nightly, which isn’t realistic for VMAF at most scales. Coverage often beats precision.

Bounded, comparable scale

A 0-to-1 range with a fixed maximum makes scores comparable across clips, codecs, and resolutions. PSNR’s unbounded decibels make cross-content comparison harder.

Spatial maps for diagnosis

SSIM is computed per window, so you can render the result as a heatmap and see exactly which regions degraded. That turns “quality dropped” into “the encoder is losing the shadow detail on the left.”

Built into the toolchain everywhere

FFmpeg, x264, x265, MATLAB, scikit-image, OpenCV, PyTorch, and every commercial analyzer implement it. Twenty years of near-universal adoption means results transfer between tools and teams.

Stable defaults

Because k₁ = 0.01 and k₂ = 0.03 went unchanged since 2004, an SSIM score means roughly the same thing everywhere. That reproducibility is rarer in quality metrics than it should be.

Limitations of SSIM

No temporal awareness

SSIM was designed for still images. It compares frame to frame independently and never sees motion, judder, frame drops, or flicker.

A stream that alternates between good and bad frames can average to a fine score while looking unstable. Mitigation: track per-frame variance and worst-case windows, or use VMAF where motion matters.

Uniform pooling hides local failures

The standard mean across windows treats a wrecked corner of the frame the same as a marginally soft one. Faces and text, where viewers actually look, get no extra weight. Mitigation: use percentile pooling, or weight regions of interest yourself.

Instability in low-variance regions

In flat areas, σ approaches zero and the contrast and structure terms become numerically unstable. Dark scenes, sky gradients, and solid backgrounds can produce misleadingly high or erratic scores. Mitigation: MS-SSIM handles flat regions better than single-scale SSIM.

Underestimates distortion near hard edges

Research on medical imaging found SSIM understates artifacts adjacent to sharp boundaries, exactly where ringing and mosquito noise appear. Mitigation: check edge-heavy content visually rather than trusting the score alone.

Blind to resolution and viewing conditions

An identical SSIM score means different things at 360p on a phone and 4K on a TV. SSIM has no concept of display size or viewing distance. Mitigation: set per-rung thresholds instead of one global gate.

Nothing about delivery

SSIM grades pixels only. It says nothing about startup time, rebuffering, or video latency, and viewers abandon streams over those long before they complain about texture. Mitigation: pair SSIM with playback QoE telemetry.

Not a distance function

Formally, SSIM doesn’t satisfy the triangle inequality or non-negativity, so it isn’t a metric in the mathematical sense. Rarely matters in practice, but it blocks certain approaches that assume metric properties.

Knowing the formula and the commands is one thing. Wiring SSIM into a pipeline that makes encoding decisions on its own is where the metric either pays for itself or becomes a number nobody looks at.

Here’s what that looks like in practice.

How to Use SSIM in a Video Encoding Workflow

SSIM is most valuable as an automated gate, not a manual spot check. A workable setup runs in six steps.

1. Build a reference clip set. Pick 8 to 15 clips of 10 to 30 seconds that cover your real content: a talking head, a fast pan, a dark scene, a grainy shot, a screen recording, a sports clip. Keep them as pristine mezzanine files. This set is your ground truth for every future encoder change.

2. Sweep settings per clip. Run your CRF or bitrate range across every clip and record SSIM for each combination. This is embarrassingly parallel, so it fans out cleanly across workers.

3. Find the knee in each curve. Plot SSIM against video bitrate per clip. Curves flatten hard past a certain point, and beyond it doubling bitrate buys 0.002 SSIM. That flattening point is your efficient operating range.

4. Set per-rung targets. Different ladder rungs deserve different gates. A 1080p top rung might target 0.96 while a 360p mobile rung targets 0.88, because nobody’s inspecting texture on a phone at 400 kbps. Match the target to what the rung is for, and let your adaptive bitrate streaming ladder reflect those decisions.

5. Pool with the worst case, not the mean. Mean SSIM is a bad gate because one destroyed 60-frame scene barely moves it. Track the 5th percentile frame score, or the worst 1-second rolling window, alongside the mean. Fail on either.

6. Gate it in CI. Run the sweep on every encoder upgrade, preset change, or parameter tweak, and fail the build if any clip drops more than a set delta. Encoder regressions are quiet. A preset change that costs 0.01 SSIM on grainy content shows up in support tickets weeks later, not in your logs.

One thing SSIM won’t catch: timing. It compares frame N to frame N, so a stream with correct pixels and a broken keyframe interval scores fine while players struggle to switch renditions. Segment structure and HLS streaming manifest correctness need their own checks.

This is also where teams decide whether to own the pipeline at all. A quality-measured encoding ladder means running the sweeps, storing mezzanines, tuning per-codec settings, and re-checking on every encoder release.

LiveAPI handles the encoding side directly, with instant transcoding, adaptive bitrate ladder generation, and HLS output delivered through Akamai, Cloudflare, and Fastly. The ladder and rendition tuning are already done. Teams that would rather ship a video feature than staff an encoding quality program tend to land here.

Tools for Measuring SSIM

For custom pipelines, GStreamer can host quality measurement inline as part of a processing graph rather than as a separate pass.

And if you’re calling out to a hosted video transcoding API instead of running your own farm, the encoding decisions SSIM informs are already made upstream. Your job shifts to spot-checking output rather than tuning every rung.

Encoder choice affects your numbers too. Hardware encoders trade quality for throughput at the same bitrate, so a switch between software and hardware encoders will move SSIM even with identical settings. Measure after the switch, not before.

Do You Need to Measure SSIM Yourself?

Running your own SSIM pipeline makes sense in some situations and is overhead in others.

Measure SSIM yourself if you:

• Run your own encoding infrastructure and tune presets
• Ship a large VOD catalog where bitrate savings compound into real money
• Are evaluating a codec migration and need evidence
• Have content types (grain, animation, screen capture) that behave unusually
• Need audit-ready quality documentation for customers or regulators

Skip it if you:

• Use a managed encoding platform that already generates tuned ladders
• Ship a modest volume where engineering time costs more than bandwidth
• Have no mezzanine reference files to compare against
• Are streaming live, where there’s no time for full-reference analysis anyway
• Have viewers reporting buffering rather than blur, which is a delivery problem

Live streaming deserves a specific note. Full-reference metrics need the source and the output side by side, which is awkward in real time.

Most live workflows measure SSIM offline on the recorded output, or on representative test streams before an event, and monitor input health during the broadcast instead.

SSIM FAQ

What does SSIM stand for?

SSIM stands for Structural Similarity Index Measure, sometimes written as Structural Similarity Index. That’s the SSIM full form in both image processing and video engineering. The name points at the core idea: measuring how well structural information survives compression.

What does SSIM measure?

SSIM measures how much a processed image or video frame differs from its original across three dimensions: luminance (brightness), contrast (variation), and structure (pattern correlation). It combines them into one score between 0 and 1 that approximates what a viewer would perceive.

What is a good SSIM value?

Above 0.95 is generally good, above 0.99 is indistinguishable from the source, and below 0.90 usually shows visible artifacts. But the SSIM range is content-dependent: grainy footage can look fine at 0.88, while flat animation can look wrong at 0.98. Calibrate against your own content.

Is a higher SSIM always better?

Higher means closer to the reference, which is usually what you want. But chasing 0.99 across a whole ladder wastes bandwidth on quality nobody sees. The goal is hitting a target that’s good enough for each rung, not maxing the score.

Can SSIM be negative?

Yes. The SSIM value range is technically −1 to 1, and negative values appear when windows are anticorrelated, like an inverted image against its original. Real encodes never land there, which is why 0 to 1 is the practical range.

What’s the difference between SSIM and MS-SSIM?

SSIM evaluates at one scale; MS-SSIM evaluates at roughly five scales and weights the results. Perceived quality depends on viewing distance and display size, so MS-SSIM tracks human judgment better and is the stronger default for video work.

Does Netflix use SSIM or VMAF?

Netflix built and open-sourced VMAF in 2016 because SSIM and PSNR weren’t accurate enough for their per-title encoding work, and VMAF is what they use for quality decisions. Plenty of teams still run SSIM alongside it as the cheap, broad-coverage check.

How do you calculate SSIM in Python?

Use structural_similarity from skimage.metrics for images, passing full=True to also get a spatial SSIM map. For video, call FFmpeg’s ssim filter from Python or use the ffmpeg-quality-metrics wrapper. For differentiable SSIM loss in training, use pytorch-msssim.

What is SSIM loss?

SSIM loss is 1 − SSIM (or DSSIM) used as a training objective for neural networks in super-resolution, denoising, and learned compression. It produces sharper output than mean squared error loss, which pushes models toward blurry averages.

Does SSIM work for comparing videos in different codecs?

Yes. SSIM compares decoded pixels, so the codec is irrelevant as long as resolution and frame count match. That’s what makes it usable for codec comparisons like H.264 against VP9 or AV1 at equal bitrate.

Getting SSIM to Work for You

SSIM earns its place as the workhorse of video quality measurement: cheap, well understood, implemented everywhere, and far closer to human perception than PSNR.

Where it falls short is temporal behavior and local failures. That’s why the strongest workflows pair SSIM’s broad coverage with VMAF’s accuracy on samples, and gate on worst-case frames rather than averages.

The real payoff isn’t the number. It’s using it to find the point where more bits stop buying visible quality, then shipping ladders that sit at that point instead of guessing.

If building that measurement pipeline isn’t where you want your engineering time going, LiveAPI handles encoding, adaptive bitrate ladder generation, and global delivery through a single API, up to 4K, with instant transcoding and HLS output ready in seconds. Get started with LiveAPI and ship your video feature without staffing an encoding quality program to go with it.