Big File Uploads

There is more than one way to get a file onto a website: a simple upload form, an FTP client, or — in years past — a Flash uploader.

As the owner of the site, with administrator rights, you can use any of them. Once you want your visitors to upload files, the options narrow. FTP is out: you are not going to hand out server credentials. Flash is out too — browsers dropped support for it in 2020.

What remains is the HTML upload form, which is about as plain as an interface gets:

Press Browse, choose your files, press Send, and up they go.

Why big files fail

HTML upload forms have a real limitation: web servers cap how large an upload may be and how long it may take. “Large” here starts somewhere around 50 MB, though it depends entirely on the server and on how fast the visitor’s connection is.

Sometimes raising a couple of PHP settings — upload_max_filesize and max_input_time — is enough. Often it is not: on shared hosting you may not be allowed to change them at all, and a slow connection will still run out of time. So how do your visitors upload big files?

The solution: chunked uploads

HTML5 brought, among much else, the ability to slice a file up in the browser before sending it. A big file can be cut into pieces, the pieces sent one at a time, and the file reassembled on the server once they have all arrived. Make the pieces small enough — a megabyte, say — and no server will object to any of them. The size of the original file stops mattering, whether it is 1 GB or considerably more.

This is called chunked upload. Implementing it properly is fiddly, but it brings three real advantages:

• Files of unlimited size can be uploaded — bounded only by the file system, or by the storage the server has left.
• Checks that run before a file is saved — its extension, or whether its name matches some pattern — need only the first chunk, not the whole file.
• Uploads can resume. If the connection drops halfway through, the upload picks up where it stopped instead of starting again.

There are technical write-ups of how to build this, but they assume solid programming experience and a good understanding of how uploads work. The practical answer for most sites is a tool that has already done it.

Further reading

• HTML5 features, including the File API: developer.mozilla.org
• General-purpose upload libraries supporting chunked uploads: Plupload and Fine Uploader
• A WordPress plugin that supports them out of the box: Iptanus File Upload