The flexibility of public cloud infrastructure allows for little to no upfront expense, and is great when starting a venture or testing an idea. But once a dataset grows and becomes predictable, it can become a significant base cost, compounded further by additional costs depending on how you are consuming that data.
Public clouds were initially popularised under the premise that workloads are dynamic, and that you could easily match available compute resources to the peaks and troughs in your consumption, rather than having to maintain mostly idle buffer capacity to meet peak user demands. Essentially shifting sunk capital into variable operational expense.
However, what has become more apparent is that this isn’t necessarily true when it comes to public cloud storage. Typically what is observed in a production environment is a continual growth of all data sets. Those that are actively used for decision making or transactional processing in databases, tend to age…