Since on the majoroty of my career, I was working with live streaming in some way, i decided that it would be a fun challange, to create a VOD/transcoding on demand system. The idea for now is pretty simple:
- Playback with only direct play or transmuxing aka. using copy to not waste any resource
- If the network slow/codec not supported by client transcode on demand
- Provide native support for ASS/SSA subtitles
- Have a custom application, which will play the media properly using libmpv. For now I’ll stick with Kodi and a simple Python script which launches MPV though.
I took some inspiration from other projects, which kind of doing the same thing like Jellyfin, Plex, Emby
The difference that I want to make:
- Support an indexed file browsing, not just browsing based on metadata (later I plan to support metadata browsing as well)
- Support ASS/SSA subtitles wherever it’s possible. I’m consuming lot of Anime series which are mostly having these subtitles. Neither Plex and Jellyfin players does not support natively
Yeah basically that’s it, as I said this is like just a fun project for me to deepen my knowledge on every aspect of the development.
Transcoding vs Transmuxing
Will shortly explain transcoding and transmuxing:
Transcoding: change the source data, so we encode the audio or video, we add filters, changing resolution, bitrate etc…
Transmuxing: The data remains the same we just change the container. For example I have a mkv file and I’m changing it’s container to HLS compatibile m4s or ts.
In this project when I say transcoding that also includes transmuxing as well, because usually I need to change the container as well to m4s or TS formats.
Risks of using only transmuxing: I noticed some of the players are not supporting varied HLS segment times, for example Kodi struggles with seeking, determining the correct segment timings. So for these occurances we can use direct play or transcoding the video properly
Stack
I want to keep everything simple as possible, I won’t build a scalable artichture, due this application will run on your home server, where it’s probably more important to have less resource usage.
Backend
Picking FastAPI (python) with SQLite database. Python is not the fastest language, but due most of the heavy job will be done by FFMPEG and the file transport handled by NGINX, it’s fine. It feels unnecessary to use, lower level language like golang.
Frontend/App
For now I’m gonna just create a Kodi plugin and a simple script which will run MPV. I also vibe coded a kotlin app as a prototype, but it’s full of slop, so won’t publish that. At least I made sure with this prototype, that the idea will work, so it’s worth the time investment to learn Kotlin and Android development.
Deployment
Simple, user will build docker image and use it. Some internals though:
- Most likely will use alpine with FFMPEG and NGINX
- Need to make sure that FFMPEG also properly will support, GPU transcoding if the user has dedicated GPU.
AI usage on the project
Due this is a fun project, and I want to deepen my knowledge, for simple CRUD (boring stuff) will use AI but for transcoding only if I really stuck somewhere.
Media playback
This is the core part of the application, as you assumed.
Short workflow:
- Client picks the file from file browser
- Create a transcoding session ID
- Extract attachments like fonts and subtitles from the media
- Client downloads these attachments and use them
- Start transcoding/transmuxing
- Return master.m3u8 to client
- Client sends back the playback status to server
- If the client stops the playback or inactive for x amount of time, stop transcoding and remove files
If we have lot of bandwidth and the player supports the target media format, on that case we can also use direct play, which will not do any transmuxing/transcoding. Direct play not ideal for web, so we need HLS.
File browser
I’m creating a simple file indexer and a watchdog which is storing the files on the database instead of spinning up HDD(s) every time when you want to pick a file. This is integrated with the API itself to keep the simplicity, and keep resource usage low. There is a disadvantage though: if there is a breaking change, unhandled exception in the service, on that case it’s affecting the API and potentially the playback.
I’m building the service in the way that it’s easily refactorable, if I want to seperate indexer from the API service.
The file indexer is quite simple: I’m using Python watchdog library, which can look for IO events. The process is not writing immediately to database, the service is persisting the data by batches every 60 seconds. The same deal is with the initial indexing, we write the database in 500 record batches.
In the database we just persist supported formats: media files, subtitles, audio files.
The directories watched determined from the media collection directories that the user provides, so we won’t scan the whole filesystem.
We also do indexing regularly, probably the best decision in this case, twice a day or if the user directly requests. We need to detect the newly added, removed files which were not detected by watchdog. When the user calls the indexing endpoint, on that case we start a FastAPI background task. It’s not easily trackable, no retry logic etc… in this case, but for now it’s fine. Later rq or Celery queues can be introduced.
Starting transcoding session
For now the endpoint which is starting the session is basically just returns a random UUID for the client and creates a session on the server, no further tasks done by this endpoint. This will be the identifier, which will be expected for future endpoints.
Extracting and downloading attachments
Due HLS stream m3u8 files cannot transport advanced subtitles and their fonts, I figured out, that I just simply extract them using FFMPEG and returning a list of links to the client, where it can download. Also I’m making sure that we return attachments from the container (for example MKV) and also beside files, example:
- my_movie.mkv
- my_movie.ass
- my_movie.srt
Transcoding
This is the core of the service. In short terms here we are calling, stopping FFMPEG. For on demand transcoding we have a special algorithm that when you seek we kill the old process and start a new one, I’ll provide an example :
The media duration is 90 minutes
- You start the playback, transcoding required, so we use H264 or H265 for encoding for video
- For audio we start an another on-demand transcoding process, we just transcoding the audio track which are used by the client
- You seek to 20 minutes, the service is checking if we have the corresponding segment file or not.
- If we have it, let the client continue. If we does not have on that case we check how much back we are lacking behind of the target, if the target segment is near (in the threshold range) we let the current FFMPEG process to transcode that segment, otherwise we stop FFMPEG and start a new process with the desired time. We await for the target segment properly written.
This is the basic idea of the transcoding process.
Now you have the general information regarding the application let me show a demo of the application using Python script + MPV
The project is not yet available, since it’s under heavy development and major features can break and change.
I have a roadmap to publish a beta at early next year.
Roadmap (2026):
- Have a plugin developed for Kodi
- Improve application security (auth/authorization)
- Store playback status for files
- Adaptive streaming
- Very basic web app
- First public release