How to Download M3U8/HLS (HTTP Live Streaming) Video Streams

Did you know (you probably already know) that a majority of video content on the web is delivered through streaming? A media file is split into chunks, and then delivers them sequentially as the video is played. Apple's HLS is one of the most widely used protocols that runs that process and if you watch a video on the internet, there's a good chance HTTP Live Streaming is behind it. HLS ensures that an entire media file doesn't have to be loaded to play a video which is inconvenient for any users loading pages and it is expensive in terms of bandwidth delivery. Apple's HLS is simply ingenious.

Although it is convenient and a much more cost effective way to transmit video, HLS makes downloading and converting the media chunks back into a playable file really difficult. It is like putting toothpaste back into a tube and if you don't know your way around command line tools such as yt-dlp, you're out of luck. Sometimes downloading video streams from public sources is necessary for editing videos and non-programmers are at a disadvantage. Luckily, Sorcerer has provided a solution to your struggles.

How to Find M3U8 Files

Go ahead and signup for a Sorcerer account and we suggest starting out with the basic subscription plan to get familiar with the tool. Now, lets circle back to the video you want to download. Without using a browser extension, it is impossible to tell if the video stream you are watching is delivered via HLS, MPEG-DASH, or any other streaming protocols. However, we can do some basic detective work to figure it out. In Google Chrome, access the developer tools via the keyboard shortcut, "Ctrl+Shift+I" and a sidebar should pop up. Then, click on the Network tab to review what resources are being requested by your browser and refresh the page.

Ideally, you want to pause the video after a few seconds to stop the network tab from being loaded with unneeded video segments. Then, we can use a few phrases to try to figure out if HLS is being used to transmit the stream. These phrases include: m3u and m3u8. We are looking for a .m3u or .m3u8 file. Type this into the filter input at the top left. HLS uses a specially formatted text file (.m3u8) to act as manifest or a map of where the video segments are locating so that the video player can request them.

If that manifest file is missing from the Network tab list, another streaming protocol is likely being used to deliver the video to users. However, if a manifest file does pop up, you may be confused as to why multiple .m3u8 files are now available in your network tab.

How HLS Works

This takes us back to how HLS works. The streaming protocol is considered an adaptive streaming protocol, which in simple terms means that it reads a user's internet speed and delivers video chunks of the most relevant quality relative to what the user's bandwidth speed can handle. When a video is converted into segments to be delivered using HTTP live streaming, the video is often transcoded into lower qualities for user's with slower internet speeds. Which means that there can be manifest file for each specific quality (480p, 720p, 1080p, etc.).

Additionally, it is common for the video, audio, and subtitle components to be split apart into separate files. The audio components are often downsized into lower bitrate files and so, manifest files are available for them as well. These files for specific formats are referred to as variants.

Finding the Variant Playlist Manifest

To get the variant playlist that points to the variant manifest files for each specific media component, we need to open up each one. A simple cheat code is searching for the phrases: index and master. Variant playlist files are often named master.m3u8 or index.m3u8 however, it is not a certainty. Right click on one of the filtered files in the network tab and click "Open in new tab" to download it. Since .m3u8 files are text files, we can use a text editor such as Notepad++ or Sublime Text to view the contents.

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PLAYLIST-TYPE:VOD
#EXTINF:10.010,
segment0.ts
#EXTINF:10.010,
segment1.ts
#EXTINF:10.010,
segment2.ts
#EXTINF:10.010,
segment3.ts
#EXT-X-ENDLIST

Looking at this manifest, you can see that it pertains to a specific variant. You can see each video segment marked with "#EXTINF:" and its length in seconds listed afterward. The segment's location are marked below where they listed. In this case, the segments don't have a full URL just their file name (stated as segmentSegmentNumber.ts), indicating that their location is in the child folder of where this variant's manifest was downloaded. Although we can technically download the stream presented with this manifest by inputting its URL into Sorcerer, it would be inconvenient as we have no idea how large the resulting file would be, its resolution, or whether or not this stream is actually a video or audio stream. Lets go back and download another .m3u8 file in Chrome's network tab so we can find the variant playlist manifest.

#EXTM3U
#EXT-X-VERSION:1

#EXT-X-STREAM-INF:BANDWIDTH=736221,CODECS="avc1.4d401e,mp4a.40.2",RESOLUTION=640x360,SUBTITLES="subs"
https://bbgvod-azure-us-east1-zenko.global.ssl.fastly.net/vod/vod/m/OTA4MDIwNQ/Q2xvdWRfNjgwMzA1/7d7832ef-3339-402d-aa32-c4c7fd343468/6/index.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=3221090,CODECS="avc1.4d4028,mp4a.40.2",RESOLUTION=1920x1080,SUBTITLES="subs"
https://bbgvod-azure-us-east1-zenko.global.ssl.fastly.net/vod/vod/m/OTA4MDIwNQ/Q2xvdWRfNjgwMzA1/7d7832ef-3339-402d-aa32-c4c7fd343468/2/index.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=1349264,CODECS="avc1.4d401f,mp4a.40.2",RESOLUTION=1280x720,SUBTITLES="subs"
https://bbgvod-azure-us-east1-zenko.global.ssl.fastly.net/vod/vod/m/OTA4MDIwNQ/Q2xvdWRfNjgwMzA1/7d7832ef-3339-402d-aa32-c4c7fd343468/4/index.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=940569,CODECS="avc1.4d401f,mp4a.40.2",RESOLUTION=960x540,SUBTITLES="subs"
https://bbgvod-azure-us-east1-zenko.global.ssl.fastly.net/vod/vod/m/OTA4MDIwNQ/Q2xvdWRfNjgwMzA1/7d7832ef-3339-402d-aa32-c4c7fd343468/5/index.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=100641,CODECS="mp4a.40.2",SUBTITLES="subs"
https://bbgvod-azure-us-east1-zenko.global.ssl.fastly.net/vod/vod/m/OTA4MDIwNQ/Q2xvdWRfNjgwMzA1/7d7832ef-3339-402d-aa32-c4c7fd343468/64/index.m3u8

#EXT-X-MEDIA:LANGUAGE="en",AUTOSELECT=YES,FORCED=NO,TYPE=SUBTITLES,URI="/media-manifest/videos/aa9fb9e4-2cad-42eb-bd8a-97f45a59983b/sub.m3u8",GROUP-ID="subs",DEFAULT=YES,NAME="English"

Now, this is what we are looking for. This is the variant playlist as it lists every single variant available for that one video. Each variant starts with "EXT-X-STREAM-INF:" and they state the metadata for each stream pertaining to its bandwidth, codecs, resolution, and its relevant subtitle track if available. The variant manifest's URL is listed below the metadata. For this specific playlist, the audio and video components are merged into one file based on the codecs metadata because it lists the audio codec (mp4a.40.2) and video codec (avc1) together. If we wanted to download an audio stream without the video component, we would have to look for the specific variant with the codec only listing the audio component. This variant playlist has one such stream listed second to last above.

Downloading the Stream

Now that we have the location of the variant playlist, you are now wondering how we can download the stream into a media file. We just download the hundreds of .ts segments that can pertain to a variant and glue them together, simple! Well, that's technically the correct process however, its nearly impossible to do this without a custom script or using an open source alternative. This could have you tearing out your hair or making you feel inadequate about your computer skills as you look for solutions to this "simple" problem.

This is where Sorcerer comes into play. We can easily input the variant playlist URL into Sorcerer's process creator to download our preferred variant or variants. Go into the dashboard and click on "Processes" at the top. Once there, click on the plus icon in the top right to start filling in the inputs. Paste in the URL, select the cloud connection you want to import the newly created file into, and click on the create button at the bottom right.

Sorcerer does the hard work of extracting each variant and it lists them for you with their relevant metadata. We even approximate the file size so you don't accidentally download a file too large. The tool separates the components into video, audio, and subtitle pages so that you can customize what is included in the outputted .mp4 file. If you only wanted to download the subtitle track, just skip over the video and audio pages and select the specific languages you need. Once you select what you need, click the create button at the bottom right and the download process will start.

Once the process starts, click on the created process to check on its status. You can see the progress on the download of each component and if it fails. Sorcerer fetches the components, muxes them into one .mp4 file, and uploads it into your preferred cloud platform. That was a lot to download one video! If you wanted to automate this, you can use Sorcerer's API to auto-select the best qualities and start the process.