FFmpeg patches PixelSmash after Jellyfin remote takeover risk

FFmpeg has released version 8.1.2 to fix a newly disclosed PixelSmash flaw (CVE-2026-8461) in the MagicYUV decoder. Security researchers say the bug can be exploited for remote code execution on Jellyfin under specific conditions—then also cause denial-of-serv

When a server administrator opens a media folder, the last thing they expect is a crafted video file becoming code. Yet that’s the risk now attached to a widely used piece of software: FFmpeg’s MagicYUV decoder—an area used far beyond just video players.

A newly disclosed FFmpeg flaw dubbed “PixelSmash” can be exploited for remote code execution on Jellyfin servers under certain conditions. It can also trigger a denial-of-service condition in applications like Kodi, Emby, Nextcloud, PhotoPrism, and OBS Studio.

The vulnerability is tracked as CVE-2026-8461. It’s described as a heap out-of-bounds write in the MagicYUV decoder and received a high-severity score of 8.8. A malicious video file in AVI, MKV, or MOV format is the route to exploitation.

Any application that uses libavcodec—FFmpeg’s core library for video decoding and encoding—is considered vulnerable. In practice, that means the problem can show up anywhere a system relies on FFmpeg to turn uploaded or discovered media into previews, thumbnails, or metadata.

PixelSmash doesn’t need an exotic setup. JFrog researchers say the flaw can be triggered when a user opens AVI, MKV, or MOV video files, browses a directory containing the file through thumbnail generation, or runs any automated media ingestion workflow.

That “automated” part matters. JFrog found that multiple popular media applications rely on FFmpeg with the MagicYUV decoder enabled—making them potential targets. Those include Kodi, OBS Studio, PhotoPrism, and the thumbnail generators used in GNOME/KDE/XFCE desktop environments.

JFrog also notes that Slack, Discord, Telegram, and WhatsApp may be susceptible because they use FFmpeg to generate server-side video previews, though they were not tested.

The supply-chain angle is hard to ignore: JFrog says PixelSmash is present in hundreds of projects that “trust FFmpeg to handle untrusted input safely,” turning a decoder bug into a cross-ecosystem weakness.

The researchers trace the root cause to how MagicYUV processes slices—independent regions of a video frame that can be decoded separately. JFrog explains that PixelSmash stems from “an inconsistency between how the frame allocator and the decoder compute chroma plane heights. ” and describes it as a one-row heap buffer overflow in the MagicYUV decoder’s slice handling.

It’s not just about a crash. In the most serious demonstrations, JFrog lead researcher Yuval Moravchick showed PixelSmash could be used for remote code execution on Jellyfin and Nextcloud instances with Movie preview enabled.

“To demonstrate the real-world impact, we achieved full remote code execution against a Jellyfin 10.11.9 media server – the second-most popular self-hosted media server (after Plex) – through its normal media library scan pipeline,” JFrog says.

The attack path Moravchick outlined is specific: “a download of a crafted MagicYUV AVI into the media library -> Jellyfin automatically triggers ffprobe for metadata extraction -> the OOB write fires -> AVBuffer.free is hijacked to system() -> arbitrary command executes as the jellyfin service user.”.

There’s a catch: Moravchick noted that the remote code execution exploit requires Address Space Layout Randomization (ASLR) to be disabled, and that CVE-2026-8461 alone does not bypass this memory protection.

In theory. the researchers say a separate information-disclosure bug in FFmpeg’s FlashSV decoder could be chained with PixelSmash to bypass ASLR. They also describe another attack scenario via torrent downloads that requires no user interaction. In that version. an attacker could seed a malicious video designed to target Jellyfin users who point the download to the application’s media library folder.

“Jellyfin’s real-time file system monitor detects the new file and automatically triggers an ffprobe metadata scan. The exploit fires during the scan – AVBuffer.free is hijacked to system(), and the attacker’s reverse shell command executes as the jellyfin service user,” the researchers say.

Even if remote code execution is blocked—whether by ASLR being enabled or by other constraints—JFrog emphasizes that CVE-2026-8461 should still be able to reliably produce a denial-of-service condition on vulnerable targets.

The immediate question for administrators is what to do next. The researchers found that Plex mitigates the risk because it uses a custom FFmpeg build in which decoders are disabled and a minimal allowlist is in effect.

FFmpeg has now fixed the flaw. JFrog says FFmpeg releasing version 8.1.2 addresses PixelSmash, and that version was released on June 17. JFrog also notes that Jellyfin updated its bundled FFmpeg version, and PhotoPrism is working to add a file format blocklist to prevent potential exploitation.

Nextcloud’s team received the report via HackerOne, but declined to address the flaw because it exists outside of Nextcloud.

JFrog says it discovered PixelSmash (CVE-2026-8461) and reported it to the FFmpeg security team on May 13. The warning from the researchers is blunt: PixelSmash’s reach is enormous because so many projects rely on FFmpeg to process untrusted media.

In the end, the fix is straightforward—upgrade to FFmpeg 8.1.2—but the exposure is bigger than a single app. A decoder bug inside a shared library can quietly travel through entire ecosystems, waiting for the moment a server decides to scan, preview, or thumbnail an incoming file.

FFmpeg PixelSmash CVE-2026-8461 MagicYUV Jellyfin Nextcloud Kodi PhotoPrism OBS Studio libavcodec ASLR cybersecurity remote code execution denial-of-service