400 Hours of Video is Uploaded on Youtube per Min How do they Add storage & Maintain ?

 

This:

lands in this:

which in turn land in those:

which are kept in places like that:

First photo - 4TB storage, though there are already 12TB models;

2nd photo - 28 of first-photo-things can get crammed in there, or 56 of 2.5″ hdds

4x28~100TB in 2U rack

3rd photo - in single “closet” you can cram 28 of 2nd-photo-thingies

2800TB of storage. - So we’ve reached a Petabyte in a rack . Even assuming you have to make redundancies, which will cut the capacity of one rack by 20%; you also keep backup physically apart - meaning 2 Racks like this, in two different server farms for 2PB of data secured.

Again, assuming based on data I found for this answer, Youtube generates around 400h videos each minute which means it’s either:

• 30GB/h of footage(all resolutions converted, based on assumption everyone uploads 4k content) this gives you a whoping 12TB/minute. , 720TB an hour and 17PB/day
• or 3GB/h (Full HD only; converted to lower resolutions as always) which effectively cuts this to around 2PB/day.

Here’s your answer - if % share of 4k uploads will rise to a 100, the YT will need up to 20 racks a day installed. As long as FHD is more popular, they are “limited” to 3–4 racks a day.

So… they’ll just buy more storage, preferably by making their own “server farms” around the globe.

They are not stupid, you know ^^ They try to plan ahead with expected storage requirements; and try to make high enough income from services/ads to be able to cover the costs;

They also aim at lowering amount of competing standards and improving compression methods: previously we’ve had flash video and native HTML5 battle; now we have H.264 and VP8/VP9 battle. It means YT still needs to convert to both H.264 and VP8/VP9 which results in around doubling the storage required.

Of course there’s a catch - newer codecs usually tend to sacrifice quality just to save space. That’s why any streaming platform, even with 4k content, offers worse quality than FullHD content from Bluray disk. At best case scenario 4k streamed will be as good as FullHD Bluray. Compromises.

• YouTube does not use ContentID to find similar videos and “stitch them together” to lower storage requirements, ContentID checks whether video is a potential copyright infringement.
• there is no compression method available for videos - codecs used are already a highly-efficient compression. If something offers compression for videos, it will most likely mean lossy recoding (lowered bitrate -> losing additional details from the footage)
• Codecs - mp4/AVC1 and webm/VP9 are used simultaneously for compatibility (only >FHD resolutions are webm-only); audio tracks are stored as separate files for most resolutions - since audio is basically limited to stereo, sampling rate, bitrate, and they decided that it’s better to store only 4x audio than 14x audio(mixed with video)
• technically YouTube is a “Cloud” service - it checks all the boxes. Yes, scalable, redundant, accessible anywhere, one-size-fits-all solution.
• YT does not delete older videos - at most, lower view count, rarely watched or watched only in certain area videos are stored in 2–3 locations; popular ones are stored all around the globe fully utilizing available YT CDN (content distribution network). That’s why just-uploaded or older videos might take a second or 3 longer to load, while popular ones start almost instantaneously - there’s many CDN copies of those in “most popular” viewing areas.

-Michał Zińczuk