Encoding is hard work and it's quicker to submit an encoding request than it is to execute that request. Any customer can flood our systems with requests for hundreds of terabytes worth of video imports in just a second, and these could all be valid requests.
To deal with peaks, we have spare capacity on standby. But given finite budgets, the headroom is finite too. That's why we'll parallely scale up machines to handle peaks beyond that available "headroom", which we can do within a minute. In that time, Jobs could be queueing up. To make sure Jobs that require a real-time feel are impacted the least, we have made a separation between high (live) priority, and low (batch) priority traffic.
- If you use direct file uploads, your Assembly's Encoding Jobs are put into our Live Queue scheduled for immediate processing. You can think of the Live Queue as our fast lane, while the Batch Queue is for trucks only. This way, we can accept both huge library conversions as well as real-time avatar resizes without the former blocking the latter.
- If you use our import Robots with more than one file import per Robot, your Assemblies are considered to be a "batch import" and are placed into our Batch Queue right away. If there are many "trucks" in traffic, this can drastically impact execution times.
- If you set
queue: 'batch'in your Steps, you can downgrade to a lower priority queue yourself (but not upgrade). You would do this when you want your large backlog of media library transcoding to not steal capacity away from your high priority Jobs — or just to be a good citizen :)
- If you send too many Live Jobs, they can trickle down into the Batch Queue as to not affect other customers' performance.
Let's say your plan comes with a Job Slot Limit of 120 (this is always per region). That means in each region you could have 12 concurrent image resizes or 2 video encodings before jobs would be trickling down into the Batch Queue.
The Batch Queue isn't necessarily slower than the Live Queue, but whenever there are more Jobs than resources available, the Live Queue is prioritized.
As indicated, the tipping point for trickling down is controlled by the Job Slot Limit. It actually defines two values:
- How many Live Queue Job Slots a customer account can claim at any given time, for all requesting IPs.
- How many Live Queue Job Slots a single IP can claim at any given time.
The single IP value is often lower and helps to protect your end users from other end users that have extraordinary uses.
To avoid being impacted by degraded performance due to this, you can do two things:
Get a higher Job Slot Limit by upgrading your plan. Higher plans have higher throughput, and you can also contact us if you're interested in custom values.
false(available in, e.g., the Uppy integration). This makes sure media processing is handled in an asynchronous way, so two-minute queue times won't block the end-user experience. As soon as the files are uploaded to us, the user is on their way. We ping your
notify_urlwhen the files are ready, and you notify your user. You can find more on this in the documentation for Assembly Notifications. This is the recommended way of integrating Transloadit and you should use it whenever you allow uploading files from a client. It is not only the best experience for your users, but it also offers the most reliability: it allows you to enable Assembly Replays in case there was a problem, because you already have a way to notify your user/systems in an indirect way.