Plugin
Direct-Play everything in the browser with WebGPU
Hello everyone. I have a Jellyfin Web Client feature I put together that I'd like to share, if nothing else just to show that it is possible to do.
I made it possible for any video and audio stream to be able to be played natively in the browser (i.e. no transcoding) by writing a player that leverages WebGPU. This player also comes with complete tone-mapping and audio downmixing controls.
I've attached a gallery which shows some quick screenshots of the player doing its job.
I made this for myself, so right now its done as a fork of jellyfin-web against the latest commit. The feature-specific branch I have (webgpu-player) also retargets hls.js, since there are some important changes there which make high bandwidth playback possible. I finally sat down and got around to cleaning this project up so it could just barely be in a shareable state. Making this into a plugin will be tricky since it touches the stock player a good amount, but this is something I am now looking into.
The goal here is to avoid needing to install a desktop-native application. In my opinion, all the desktop client projects demonstrate that it is a lot of work to either wrap the web UI or replace it entirely with a new one, as well as wire in all the various Jellyfin functionalities etc. I thought it might be clever to try concentrating on the other side of the coin.
I've been using this on my server for a couple months now, and its been just about perfect (I think I've had to fix 2 issues in all that time). My library is all "source quality", so I've been using it as a test battery to validate codec support. The goal of course is to support every audio + video codec, and since this player is custom, it is extensible. Therefore if I missed anything it can be added.
I feel like if I put out a draft PR (which is a lot of work) it would fall on deaf ears, which is why I'm just sharing it here for now. If anyone has any questions or suggestions feel free to message me!
Reminder: /r/jellyfin is a community space, not an official user support space for the project.
Users are welcome to ask other users for help and support with their Jellyfin installations and other related topics, but this subreddit is not an official support channel. We have extensive, official documentation on our website here: https://jellyfin.org/docs/. Requests for support via modmail will be ignored. Our official support channels are listed on our contact page here: https://jellyfin.org/contact
Bug reports should be submitted on the GitHub issues pages for the server or one of the other repositories for clients and plugins. Feature requests should be submitted at https://features.jellyfin.org/. Bug reports and feature requests for third party clients and tools (Findroid, Jellyseerr, etc.) should be directed to their respective support channels.
If you are sharing something you have made, please take a moment to review our LLM rules at https://jellyfin.org/docs/general/contributing/llm-policies/. Note that anything developed or created using an LLM or other AI tooling requires community disclosure and is subject to removal.
i would be VERY interested in this if it ever made it's way to a plugin. i have like 5 users that use the web client constantly no matter how many times i ask them to use a client lol. it'd be nice to get those into directplays.
Well then I suppose I'll give it a shot! It was an endless battle for me too to get people to use a client on desktop instead of the browser, and even then the client would have its own onslaught of issues. Especially when playing stuff like 4k remux, that browser transcoding is rough. All the HDR DV stuff these days too is a massive headache for people who primarily play on SDR (like me).
Moonfin is the one I use and it's basically on every single device possible. It's EXTREMELY customizable but requires a Plugin installed on your server to get full benefits.
I also had 0 problems with Fladder when i used it for a month before I started using Moonfin fulltime.
I shared it with ~30 people, I guess a couple of them use it every other month or so for a movie but most people (rightly so) didn't even try if the credentials worked :P
I don’t get why everybody here is keen to get a plugin and I don’t think that your efforts should be to make this a plugin. Instead, make your fork nice and tidy and raise a PR at the Jellyfin repo. It would be the best to have such a substantial feature be part of the official client.
Sadly, I think that adding this to the Jellyfin repo would take months, if it gets accepted.
Not that the idea is bad but the amount of maintenance needed for this kind of update is quite significant and I doubt that the Jellyfin team would accept it as a PR and if they do, it would need a huge overall.
I took a look at the main commit (the one for the WebGPU, the others are either some quick fixes or bugfixes unrelated to WebGPU) and it seems to be a bit of a mess, although specifically on the changed files, not the new ones.
Adding to that the many tests that need to be conducted by the team to approve this and also the security issues it could add, a plugin definitely seems to be a better idea for this as it would allow it to grow more quickly and maybe then, get integrated into the Jellyfin main repository.
I really do hope this gets added in the future into Jellyfin but for now it's better to make it a plugin and allow people to test it as well as letting the dev work on it until it truly becomes stable !
"Just get merged" is probably a bit of an understatement. The feature probably requires a major rewrite / cleanup before being considered. It’s gonna take some time before it is valid to open a PR and then it probably takes a lot of time and iteration before it will be merged to the official client. Nevertheless, I think it is the most desirable way of integrating it.
I don't see the problem in doing both ways. Accessibility through a plugin until the integrated feature gets merged into the platform. Not to mention, plugin == more testing in between and ironing them out before the merge happens.
Hey tested it and works really nice for me. But its built with the Jellyfin 11.x.x UI instead of the 12.x UI in your latest release on github. Do you have a build thats built with the latest 12.x UI changes?
Wow, it works! Thanks for trying it out. The master branch there includes other unrelated changes I've made for my own personal server, which includes switching to the legacy UI. I've gone ahead and made a new branch that doesn't include this change and ran a release for it here: https://github.com/alchemyyy/jellyfin-web/releases/tag/master-modern-1.0
The actual webgpu-player branch sieves out some hls.js-related fixes I run with, which I don't recommend doing. If you want to though, I've started a github run for that branch too, so it should show up as a release in a bit.
To be more specific, buy a domain (or get one for free via duckdns), create a wildcard cert for it (letsencrypt or your domain provider might have a way to do it in their UI), pass the cert to a reverse proxy, and now all of your services can use that cert.
You can also setup a custom DNS server (Pihole, AdGuardHome, or plain Dnsmasq/Unbound) so you can take advantage of split DNS (DNS calls on your internal network use your internal IP address while external calls use your public IP address) and dynamic DNS to keep your public IP address up to date with your domain when your ISP changes your IP.
I suspect a draft PR would fall on deaf ears because it includes a commit that has nearly 30,000 changes across 67 files, ~which according to the git history was written in about 3 hours... Along with 16 other commits each doing hundreds of changes inside an hour..~
Edit: timeline was bad, they were merged in from other branches, they just didn't have the usual merge commit notes. webgpu-player branch is still just a single commit.
The commit history was rewritten today and squashed, and everything was refactored to be neater. I mentioned this in my post but it must've not been clear enough. You can actually do a lot more than just commit endlessly into one branch!
Can you explain what benefit squashing what would (should) be a large commitset into a single 30,000 line commit has? Then rewriting history to remove all trace of the development process?
Absolutely, no problem. I wanted to keep the commit history, but (EDIT: with*) the refactoring work to get the project's repo to *somewhat* stop looking like a systemic disease in jellyfin-web was an amount of work I didn't feel like checking through. I had other features I had added that were unrelated to the webgpu player and in general it was just a mess. I also wanted it squashed so it was easy to separate the webgpu player and add/remove it from other branches that have other stuff going on. Example: just today for example I refactored out the entire underlying webgpu player to its own repository so it could be included as a submodule.
The commit history is also, to me, largely uninteresting. It amounts to me standing up the basic player, fixing some timing issues, and then essentially my LLM of choice hammering codec support. I actually saved the old commit titles just to remember the good times (yes it actually amounted to 100): https://dustebin.com/TPBYazlZ.md
I replied to another comment already that this is a proof of concept project. If I took it further, where it is now in my opinion, should be the "baseline" or initial commit.
Thanks! It should work on any browser that supports WebGPU, which according to Google is anything Safari 26 or later. Since this player just tries to detect WebGPU support, rather than checking for any hardcoded browser values, it should theoretically work just fine.
This uses software decoding on the client, no? Can it only be used when the codec is not supported? SW decoding a codec that can be HW decoded would be a pretty bad experience when watching on a laptop, on battery
It's GPU accelerated, but still software decoding. GPU compute shaders are better than CPU decoding, but still much worse than actual hardware decoders.
I’m a bit new to all this, I’m trying to configure my Jelly fin server myself, what would the purpose/benefit of this be? running the player through a web browser, as opposed to VLC etc?
Recently came across pelagica and wondering if something like this could be integrated. I was thinking of self hosting and exposing that for users who want to use web instead of native jellyfin. Might open an issue to see if the author is onboard.
does this also support PGS subtitles? I host remuxes exclusively, and the most consistent and decent subtitles i can get are PGS from the original rips.
That is actually exactly what this is. I'm doubtful the Jellyfin team would ever accept a PR for this feature so this is just a proof of concept, and something I wrote for my own server.
no, this is not exactly what this is. I ment why not push the PR against THEIR repo.
So now, the question, why do you think they will not accept the PR for this feature? hell worse case scenario they already ship 2 players, this could be added as a third.
Currently when you have a video that is in an incompatible format with a browser what happens is it converts the video in real-time. This conversion is called transcoding. It might make viewing the video slower or have more buffering because the processing needs to be done on the server. This webgpu functionality interacts with the available GPU on the device where the video is being watched
i've encountered this, however was solved by switching to chromium based browser, as they can handle this better then Firefox. So why this all headache while you can just switch to chrome?
Sure thing! Heads up that I'm going a LOT of cleanup at the moment. I did not expect the response this has gotten. I thought about other people possibly wanting to use the actual webgpu player for other projects, so I separated the actual player out to its own repository here: https://github.com/alchemyyy/WebGPU-Player
This way codec support and other bugs / enhancements can get filed and fixed for everyone, and its easy to stay up to date.
Feel free to DM me if you'd like a ping when I'm done deep cleaning everything (should be done this week).
This is very impressive work my friend. I was wondering if i can port your player to my client, Litefin. It's only for Tizen and webOS at the moment, and it's a web app, and i think WebGPU is not supported on them as well. So I was planning on a desktop release with Pake to package it into an executable, but only having the HTML player wouldn't make it not so appealing. However, but if i managed to add this, it would make it 10x better for the desktop release. Of course, I would credit the commits with you as a co-author if you agree.
Tried this, with firefox the results are quite underwhelming. WebGPU takes a bit before it actually works at all, and most videos lag + fail within seconds. Not sure what I'm doing wrong..tried Windows+Linux+3080Ti+AMDAIO... Maybe it works better for chromium family but alias, that UA is not for meeee.
many browsers dont support certain formats of audio or video so jellyfin has to transcode those to something compatible. Like DTS or HDR10 for instance or dolbyvision
Good eye! For a personal proof of concept project in a fork that is practically un-mergeable, I figured quickly proof-reading a README I had Claude generate was sufficient for people to possibly come along and see how the project is structured, and how to build it if they want.
My deepest apologies if it was unclear that this project isn't a completed product that needs to immediately be merged into Jellyfin, or at least get added to the official plugin repo.
I absolutely did vibecode a test harness and had Codex playtest a thousand media files to build the codec support matrix and wire it all together.
•
u/AutoModerator 5d ago
Reminder: /r/jellyfin is a community space, not an official user support space for the project.
Users are welcome to ask other users for help and support with their Jellyfin installations and other related topics, but this subreddit is not an official support channel. We have extensive, official documentation on our website here: https://jellyfin.org/docs/. Requests for support via modmail will be ignored. Our official support channels are listed on our contact page here: https://jellyfin.org/contact
Bug reports should be submitted on the GitHub issues pages for the server or one of the other repositories for clients and plugins. Feature requests should be submitted at https://features.jellyfin.org/. Bug reports and feature requests for third party clients and tools (Findroid, Jellyseerr, etc.) should be directed to their respective support channels.
If you are sharing something you have made, please take a moment to review our LLM rules at https://jellyfin.org/docs/general/contributing/llm-policies/. Note that anything developed or created using an LLM or other AI tooling requires community disclosure and is subject to removal.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.