“Our solution combines caching, deduplication, erasure coding, and sparse loading to take advantage of our needs. Without adding any customer visible complexity…we were able to achieve our scale and cold-start latency goals, while having significant headroom for future scaling.”
Brooker, Marc, Mike Danilov, Chris Greenwood, and Phil Piwonka. 2023. “On-Demand Container Loading in AWS Lambda.” ArXiv [Cs.DC]. arXiv. http://arxiv.org/abs/2305.13162.
Mastodon Source 🐘
“Approximately 80% of newly uploaded Lambda functions result in zero unique chunks, and are just re-uploads of images that had been uploaded in the past. This appears to be primarily driven by automated testing and deployment (CI/CD) systems. Of the remaining 20% of functions that create at least one unique chunk (and therefore aren’t just trivial reuploads), the mean upload contains 4.3% unique chunks, and the median 2.5% unique chunks”