Media and documents

An encoder (often FFmpeg, or a hosted service) turns one source video into several renditions, for example 1080p, 720p and 480p, and slices each into segments a few seconds long. Plain-text playlists with the .m3u8 extension list the segments, and a main (multivariant) playlist lists the renditions. The player reads the playlists, then downloads segments one after another, choosing the highest quality the connection can sustain and switching as conditions change.

Because it is just files over HTTP, HLS works with any web server or CDN and caches well. Apple introduced it in 2009, and it is published as RFC 8216. The same approach carries live streams, and Low-Latency HLS brings the delay down to a few seconds. Safari and recent versions of Chrome play it natively; other browsers use a JavaScript player such as hls.js or Video.js, built on Media Source Extensions.

MPEG-DASH is the main open alternative, and DRM systems such as FairPlay and Widevine protect paid content. The hard parts are encoding every upload into several renditions and paying for storage and bandwidth, which is why hosted services such as Mux, Cloudflare Stream and Bunny Stream, or AWS Elemental MediaConvert with a CDN, are common.

Video.js wraps the browser's video element in a skinnable, accessible control bar with captions and subtitles, keyboard shortcuts, fullscreen and picture-in-picture, plus a plugin system for ads, analytics, quality menus and playlists. Its built-in streaming engine plays HLS and DASH even in browsers that cannot do so natively. The project began in 2010 and is widely used by media sites and video platforms.

Version 10 is a ground-up rewrite, made together with the teams behind Plyr, Vidstack and Media Chrome, with much smaller bundles and first-class React and TypeScript support; earlier versions remain widely deployed. Alternatives include Google's Shaka Player, hls.js on its own for streaming, or the player that a video host provides.

Everything happens inside an AudioContext, where you connect nodes into a graph: sources (an oscillator, a decoded sound file, a microphone stream), processing nodes (volume, filters, delay, reverb, stereo panning, compression) and a destination, usually the speakers. An AnalyserNode exposes frequency and waveform data for visualisers, and AudioWorklet runs custom processing code on the audio thread, so busy page scripts do not interrupt the sound.

It is lower level than an audio element, which is enough for simply playing a file. Browsers only let a page start sound after a user gesture such as a click, so an AudioContext begins suspended until then. Libraries such as Tone.js (music and synthesis) and Howler.js (game and interface sounds) build on it.

A canvas element and its 2D context offer drawing commands for rectangles, paths, arcs, gradients, images, text and transforms, plus direct access to the pixel data. The browser keeps only the resulting pixels, not the shapes, so moving something means clearing and redrawing, usually every frame with requestAnimationFrame. The same element hosts WebGL and WebGPU for 3D, and OffscreenCanvas lets drawing run in a web worker.

It is used for charts, games, image editors, signature pads, photo filters and exporting pictures with toDataURL or toBlob. Because the content is only pixels, screen readers and search engines cannot read it without text alternatives, and working out which shape a click landed on is up to you. SVG suits a smaller number of interactive shapes, and libraries such as Fabric.js and Konva add an object layer on top of canvas.

Fabric.js keeps an object model on top of the canvas: every rectangle, circle, path, text box or image is an object with properties, and the library draws selection handles and takes care of dragging, scaling, rotation, grouping, layering and free drawing with brushes. It imports and exports SVG and can save a whole scene as JSON to load again later, which is what design tools need.

Typical uses are T-shirt and product customisers, banner and meme makers, whiteboards, image annotation and certificate or label designers. It runs in the browser and, through node-canvas, in Node for rendering on a server. Konva is a similar library with official React bindings (react-konva); for 3D, Three.js is the usual choice.

Raw WebGL, the browser's low-level 3D interface, needs a lot of code just to draw a lit cube. Three.js wraps it in familiar ideas: a scene holds meshes (a geometry plus a material), lights and cameras, and a renderer draws the scene into a canvas every frame. It includes loaders for 3D models (glTF is the usual format), physically based materials, shadows, post-processing effects and camera controls, and it also offers a WebGPU renderer.

Ricardo Cabello (mrdoob) started it in 2010, and a large ecosystem has grown around it: React Three Fiber lets React developers build scenes as components, drei adds ready-made helpers, and physics engines such as Rapier or cannon-es plug in. Babylon.js is the main alternative, with more built in for games; for flat 2D graphics, the Canvas API or SVG is lighter.

The hard parts are rarely the code. Models and textures must be optimised (compressed, with fewer polygons) to keep downloads small, and scenes need testing on low-end phones, where the graphics chip and battery limit what runs smoothly.

sharp wraps libvips, a C image library that processes pictures in small chunks, so it is quick and uses little memory even on large files. It reads JPEG, PNG, WebP, AVIF, GIF, TIFF and SVG, and can resize, crop, rotate according to camera EXIF data, blur, sharpen, composite watermarks, strip metadata and write modern formats, all through a chained API such as sharp(input).resize(800).webp().

It powers image optimisation in many tools, including Next.js when self-hosted, Astro and Gatsby. Prebuilt native binaries cover the common platforms, and it runs under Node, Bun and Deno, but it needs a server runtime: it does not run in the browser, and edge platforms such as Cloudflare Workers usually rely on their own image services instead.

ZXing ('zebra crossing') decodes 1D barcodes such as EAN-13, UPC-A, Code 39 and Code 128 and 2D codes such as QR Code, Data Matrix, Aztec and PDF417, and it can generate them as well. It began at Google and powered the well-known Barcode Scanner app on Android. Ports include zxing-js for JavaScript and zxing-cpp, a C++ version that also compiles to WebAssembly for fast scanning in the browser.

The original Java project and the JavaScript port are now in maintenance mode and accept only bug fixes, so new projects often use zxing-cpp builds, the browser's BarcodeDetector API where it is available, ML Kit on Android or Apple's Vision framework on iOS. Scanning quality depends as much on the camera, focus and lighting as on the library.

pdf-lib is written in TypeScript with no native parts, so it runs in browsers, Node, Deno and React Native. You work page by page: add a page of a given size, draw text, lines, shapes and embedded PNG or JPEG images at x and y coordinates, embed custom fonts, fill and flatten form fields, copy pages between documents, and set metadata.

There is no layout engine: beyond basic line wrapping you place everything yourself, and long text does not flow onto new pages. It cannot display a PDF either (PDF.js does that). It suits templates designed elsewhere, such as stamping a name onto a certificate or filling in an official form. Development of the original package has slowed, and the actively maintained fork @cantoo/pdf-lib adds encryption, full SVG drawing and other fixes.

react-pdf wraps Mozilla's PDF.js, the engine behind Firefox's built-in PDF viewer. A Document component loads a file from a URL or from data, and Page components render each page to a canvas, with optional text and annotation layers so text can be selected and links clicked. The PDF.js worker has to be configured so that parsing runs off the main thread.

It displays PDFs; it does not create them, and its similar name often causes confusion with @react-pdf/renderer, which generates PDFs from React components. Toolbars, search, zoom and thumbnails are yours to build, or you can use PDF.js's own viewer or a commercial viewer SDK. For long files, rendering only the pages on screen keeps memory use down.

Instead of HTML elements you use its own building blocks, such as Document, Page, View, Text, Image and Link, styled with a subset of CSS and laid out with flexbox through the Yoga layout engine. Text wraps and flows across pages on its own, headers and footers repeat with the fixed prop, and page numbers come from a render function. Custom fonts are registered once, and SVG drawing is supported.

It runs in the browser, with a preview component and download links, and in Node, where renderToFile, renderToStream or renderToBuffer build files on a server. It does not render HTML or Tailwind classes, so existing web components cannot be reused directly, and very large documents take a while to build. When a PDF must match an existing web page exactly, a headless browser such as Puppeteer is the usual alternative.

John Gruber created Markdown in 2004, with Aaron Swartz, so that text would stay readable before any conversion. The original description left many edge cases open, so CommonMark defined a precise specification in 2014, and GitHub Flavored Markdown adds tables, task lists, strikethrough and automatic links on top. Front matter, a YAML block at the top of a file, is a common add-on for titles, dates and tags.

It is used for READMEs, documentation, blogs, notes apps, chat messages and the output of AI models. Libraries such as marked, markdown-it, remark and react-markdown turn it into HTML. Because Markdown allows raw HTML, anything written by untrusted users should be sanitised after conversion to prevent XSS. MDX extends it with components.

An MDX file is compiled to JavaScript. Headings, lists and links are written as ordinary Markdown, and between them you can import and place JSX components such as a chart with its data, use JavaScript expressions in curly braces, and export values. It works with React, Preact, Vue and other JSX runtimes, and frameworks such as Next.js, Astro, Docusaurus and Nextra support it; remark and rehype plugins add GFM tables, maths and syntax highlighting.

It is stricter than Markdown. Raw HTML must be valid JSX, characters such as < and { need escaping, and indented code blocks and angle-bracket autolinks are not supported, so converting existing Markdown takes some clean-up. Because MDX compiles to code, content from untrusted users should never be compiled as MDX.

highlight.js turns the text inside pre and code blocks into spans with classes such as hljs-keyword and hljs-string, which a theme stylesheet then colours. It ships grammars for nearly 200 languages and a wide choice of themes, can guess the language when none is given, and runs in the browser, in Node or in a build step. Importing only the languages you need keeps the bundle small.

It only highlights: line numbers, copy buttons and editing come from plugins or your own code. Prism is a similar lightweight option, and Shiki uses the same grammars as VS Code for more accurate colours, usually at build time. For editable code, an editor such as CodeMirror or Monaco is the right tool.