Whether you were building a web site or an application, hosting choices used to be about bandwidth, latency, security and availability (as well as cost), with content delivery networks (CDNs) handling static assets and application delivery networks relying on load balancing for scale.
All those things still matter, but there are also many more choices to take into account — from the backend implications of frontend decisions, to where your development environment lives. CDNs have become complex, multilayer distributed computing systems that might include distributed databases, serverless functions and edge computing. What they deliver is less about static (or even dynamic and personalized) assets and more about global reach, efficiency and user experience.
In fact, the term CDN may not always be that helpful any more, because modern “CDNs” are increasingly handling not just content delivery, but also API management, distributed security and even AI inference at the edge.
It may be more helpful to think of the modern “CDN” as the front door for computing infrastructure.
Platforms like Cloudflare and Fastly describe themselves as “connectivity clouds” and “edge clouds” to reflect their expanded capabilities. (Vercel bills itself as a “frontend cloud.”) On the other hand, traditional CDN providers like Akamai offer compute in their infrastructure to compete with hyperscale cloud providers like AWS, Azure and GCP. Those companies also offer services like load balancing, access management, dynamic site acceleration, rate limiting, web application firewalls, TSL offloading, DDoS protection, API security, bot mitigation and geoblocking — not to mention traditional content delivery features.
With the line between CDN, application platform and cloud computing so blurred, and with ongoing changes in the size and type of what content gets delivered, it may be more helpful to think of the modern “CDN” as the front door for computing infrastructure, whatever that looks like.
Even the traditional CDN has evolved from caching the bandwidth-gobbling images and videos on a site to handling streaming and delivery generally, whether that’s a new game download, dynamically rendered 4K textures streaming in an open world game, or supporting e-commerce sites on Black Friday, Akamai VP of products Jon Alexander told The New Stack.
“Bulk delivery: video downloads, software, the unidirectional push of content out to consumers updating their PlayStation or Xbox, watching the video,” said Alexander. That still requires low latency and a large footprint in key regions to deliver faster traffic throughput.
But CDNs are equally important for delivering apps and digital experiences where you care more about being close to a specific set of users — or collecting data from a specific set of devices, because they’re no longer only about broadcasting or even narrowcasting content. You might also be collecting that data from sensors, IoT devices or user systems; Alexander notes that some banks are using distributed databases for real-time payment transaction data. That requires not just good bandwidth across the internet backbone, but also a lot of connections at the edge to handle very different traffic patterns — shifting from large file delivery to smaller, more frequent API calls, real-time data streams and small islands of compute.
“The average object size on our network has decreased year over year, so as requests are going up, the payloads are reducing.”
– Jon Alexander, Akamai VP of products
More of the data Akamai handles is now bidirectional streams. “They’re increasingly chatty; very high request rates, often with much smaller payloads. The average object size on our network has decreased year over year, so as requests are going up, the payloads are reducing. We’re not moving around huge game files anymore. It’s weather updates, it’s beacons.”
In fact, one of the reasons for the shifts in what CDNs offer is this change in payloads, which also changes the business model of running a CDN because what organizations are willing to pay a CDN for has changed along with the technologies they’re looking for. “It’s very hard to have a sustainable, chargeable model for APIs, because they are super light by design,” APIContext CEO Mayur Upadhyaya points out. One reason there’s less and less separation between being a delivery platform and a compute platform may be because “compute becomes the commodity you can charge for.”
For developers, Alexander suggests, this convergence of CDN and cloud computing into a distributed architecture “gives them options around how and where they run their code, without increasing the toil or complexity.”
Much More Than Hosting
CDNs (and caches) are key to using static site generation tools like Hugo, Jekyll and Eleventy for building web applications with client-side rendering of APIs via JavaScript (often referred to as “Jamstack” development), because making even a small update republishes the entire site — so there’s no risk of mixing updated and stale content on a page.
Although the majority of developers are still building infrastructure-centric applications to run in a specific region, Alexander also points at the rise of serverless and edge-centric frameworks allowing developers to abstract infrastructure from where the code physically runs.
“Your code runs anywhere or everywhere, and so does the tooling, the observability and the automation that goes with it. You have the potential now to run the application in many places, have your data in many places, and to give an incredibly interactive experience. Dynamic generation of personalized content is now feasible at reasonable latencies, so having a personalized condition of a commerce page load in one or two seconds is now very viable with distributed databases at the edge.”
Increasingly, apps need to be global, but to still feel like they’re local. The modern internet works at a global scale, and that changes the role of the CDN as well. Instead of focusing on regional and metropolitan application delivery, they need to deliver at a much larger scale, routing and storing content wherever there’s demand. Instead of offering pure content caches, the CDN becomes a dynamic, self-reconfiguring, self-repairing network providing edge access to applications and services. It also needs to both respond to and take advantage of cloud availability zones, routing user data to live services in the event of a failure.
Enter the “frontend cloud”: an “edge-distributed, incredibly fast, globally accelerated deployment layer with integrated serverless functionality.”
– Mathias Biilmann, Netlify CEO
A whole landscape of development frameworks — from simple static site generators to complex tooling like Next.js — have grown up around the idea of what Netlify CEO Mathias Biilmann calls a “frontend cloud”: an “edge-distributed, incredibly fast, globally accelerated deployment layer with integrated serverless functionality.”
That’s a long way from “hosting plus a CDN,” he argues. “Hosting is: Where do you put the files, where do you put your server? But that’s not enough to operate in production. You need to have security. If you want to offer a competitive user experience, you need to integrate the concept of a CDN or an application delivery network, you need to be able to get very low ‘time to first byte’ and get things in front of customers really fast. You need things like image CDNs, resizing and so on that you take for granted to optimize the frontend performance and responsiveness, and you need a backend for the frontend layer with serverless functions and some level of cloud storage.”
Converging these different services and layers of compute, storage, networking, security and delivery requires integration, automation and logging to allow development teams to iterate quickly, especially if you’re using services like identity and authentication, e-commerce or headless CMSes.
“Regardless of where your hosting is, you need a local developer experience for your developers …”
– Biilmann, Netlify
“You’ve got to have a way to roll back mistakes; you’ve got to have not just a production environment, but test environments and staging environments,” says Biilmann. “Regardless of where your hosting is, you need a local developer experience for your developers that makes their day-to-day development experience map cleanly to that hosting, so there’s not a discrepancy once they push it live, and so it’s easy to get set up with all the right secrets and environment variables. And around all of that, you need a layer of insights and observability: How does that hosting and automation layer talk to your normal observability platform? How can you make sure that you can see anomalies over time and get alerted early to anything that broke due to a deploy, and associate the two easily?”
Although platforms like Netlify and its competitors offer a wide range of services for both frontend and backend, taking advantage of this approach requires architecture that decouples the frontend layer from the backend. Biilmann maintains that choosing multiple services from a single provider should be about performance, or the benefits of convenience and integration, rather than a technical limitation.
“We used to be in a world where you would buy a CDN for one part of it, you would buy a hosting provider from some different parts of this, you would buy a CI/CD platform for part of this — and have to build all of this yourself. You would have to do a ton of instrumentation manually to figure out [connections] between all of these, and the security layer you would buy separately, and they would each be operated by a different team, which in itself often creates conflict.”
“Queuing and the way you wire up applications is going to be critical, because these things do live in lots of different places.”
– John Engates, Field CTO at Cloudflare
This shift also allows CDNs to expose some of their traditional functionality to developers in ways that help them deliver a better user experience. Using tools like Cloudflare Workers, you can build tools that route traffic, not only to geographically appropriate servers, but also to servers that have available capacity, using event-driven functions to respond as the workload changes. Or you can use information about a device to, for example, route translated content based on browser language settings, without having to have that same content duplicated on every edge cache. You can even put different parts of what used to be your server in different locations — an approach Cloudflare’s Sunil Pai dubs “spatial compute.”
Even getting to that extreme level of disaggregation, applications will rely on APIs and cloud services and data storage, whether that’s S3-style object storage or relational databases. “Queuing and the way you wire up applications is going to be critical, because these things do live in lots of different places,” explains John Engates, Field CTO at Cloudflare. “You’re going to have to have ways to orchestrate applications in places that are maybe distinct from where your core applications live, and you’ve got to connect all this stuff together.”
Don’t expect the same level of functionality you get from Azure or AWS, though. Instead, CDN-hosted compute — like Cloudflare Workers — have options to customize the content for where the user is and what device they’re using, as well as ways to configure software-defined networking.
Distributed and Cloud Native
One of the biggest drivers for the change in how content is delivered is simply that we’re building applications differently. Early CDN architecture was designed for monolithic applications that could be fronted with transparent caches, so they could be used to manage static content. “The first era of CDNs was just caching things so we could scale the performance and do it globally,” Cloudflare’s Engates remembers. But the days of putting a handful of NetApp Filers in front of servers (when mechanical drive speed was as much of a physical limit as the speed of light) are long gone.
“Now smart developers and engineers are building out full-fledged application platforms on top of what originated as the CDN,” said Engates.
CDNs are also having to respond to the underlying changes in modern application architectures.
Many applications are now global, running across multiple data centers and clouds, with content tailored for different regions and routed appropriately, down to monitoring the state of the BGP tables to ensure that it’s delivered as quickly as possible. Developers write code that consumes not only their own services, but needs to work with third-party APIs, which often come with their own CDNs and API management layers. Wrapping services like Shopify and Stripe inside an application means taking a dependency on their infrastructure, as well as their choice of CDNs — something it’s a lot easier not to think about.
CDNs are also having to respond to the underlying changes in modern application architectures, which in some cases can make them easier to manage. Could a CDN be part of a Kubernetes service mesh? Declarative tools for software-defined networking are a logical place for a programmable CDN to meet modern code, especially if you need to consider increasingly complex regulations and compliance issues covering everything from GDPR to where post-quantum encryption is mandated.
“Developers should not have to be experts on how to scale an application; that should just be automatic. But equally, they should not have to be experts on where to serve an application to stay compliant with all these different patchworks of requirements; that should be more or less automatic,” Engates argues. “You should be able to flip a few switches and say ‘I need to be XYZ compliant in these countries,’ and the policy should then flow across that network and orchestrate where traffic is encrypted and where it’s served and where it’s delivered and what constraints are around it.”
“People can’t build the application that they want, they can’t optimize, because of some of these taxes that are added on moving data around.”
– Alexander, Akamai
Along with the physical constraint of the speed of light and the rise of data protection and compliance regimes, Alexander also highlights the challenge of costs as something developers want modern CDNs to help them with. “Egress fees between clouds are one of the artificial barriers put in place,” he claims. That can be 10%, 20% or even 30% of overall cloud spend. “People can’t build the application that they want, they can’t optimize, because of some of these taxes that are added on moving data around.”
Update patterns aren’t always straightforward either. Take a wiki like Fandom, where Fastly founder and CTO Artur Bergman was previously CTO. “If you think about a wiki, the content doesn’t change very often, but when it changes, it is to change right away, because it’s collaborative editing.” One of his reasons for starting Fastly was that although putting the initially U.S.-centric site’s images on a CDN dropped access times for UK customers from 22 seconds (a statistic that has him reflecting that “it’s a miracle we had any users at all”) to 11, it took running a point of presence and caching the articles in the UK with a cache invalidation time of 150 milliseconds to get page load time down to two seconds. “And surprise, surprise, our usage in the UK went up, because now people could actually consume the product.”
Many services will face similar challenges as they grow. “Ultimately, the driver is how to reach global audiences, and you have two choices,” he suggests. “You can either embrace the edge or you can spin up your application on every GCP or AWS location around the world. Managing all those duplications of instances and data is expensive: or you can embrace edge and only move the things that matter to the edge.”
Putting the right content in the right abstraction layer reduces complexity and cost at your origin source. “Any piece of content that is reusable between users and doesn’t change every single moment should be cached and served from a server reasonably close to the user, because it’s always going to be more efficient, cheaper and faster to do that than to go around the world and run a database query somewhere.”
“You don’t need to cache everything: You need to cache the things that most people use most of the time.”
– Artur Bergman, Fastly founder and CTO
That works for both static and dynamic content on GitHub (which uses Fastly as a CDN) but also for e-commerce sites, which might be able to serve 80% of inventory queries from cache, or for retail point of sale, where using the edge avoids having to reconcile multiple copies of the truth.
“You don’t need to cache everything: You need to cache the things that most people use most of the time,” said Bergman.
The edge might have started as a necessity, but Bergman encourages more developers to think about the benefits — although he notes that this is easier with platforms like Magento — that have plug-ins and modules integrating CDN options like active cache invalidation into the pipeline so your CMS never gets stale content — than when developers have to implement it themselves.
“The edge is still viewed as something only a few developers should have access to: that it isn’t really part of the core architecture, it’s something you put there because you have to do. People like to cache behind their app server next to the database, and they are not generally very familiar caching in front of their application, between the application and end user.”
Too often, developers see the boundaries of their code as the edge of their own network, not the edge of the entire network they’re using to deliver their applications. But the shift to building distributed applications, often event-driven and using messaging technologies, means thinking about not just north/south connectivity from the server to the end user, but also the east/west traffic that used to be inside one data center but now needs to move across geographic replicas and even across clouds.
CDNs need to facilitate these connections between nodes, using their caches to help ensure consistency across an application. One option is hosting modern message queues, managing clusters and using their networks to ensure rapid distribution of events to queues. Using CDNs for this east/west connectivity should help keep cloud networking costs to a minimum.
“Using the edge gives you much more flexibility in adapting to the new requirements you discover as you keep developing.”
– Bergman, Fastly
This is complicated by the types of proxy, though; CDNs need to support API access to servers, running reverse proxies alongside traditional user-facing caches. While some of this functionality is provided by cloud networking tools, integrating API management into CDN infrastructure will allow developers to get the most from their networking infrastructure, putting all the relevant proxy and cache functionality in one logical layer. This simplifies future-proofing, Bergman believes. “Using the edge gives you much more flexibility in adapting to the new requirements you discover as you keep developing.”
That might include managing migrations between cloud providers to save money, he suggests. That way you can update the DNS to point at the new location, but the CDN will use its existing cache to handle any requests that come in before the data in the new location is fully cached. “They continuously migrate behind the scene, without the user knowing.”
Different services have made different choices about how to build out their infrastructure, reflecting what content they’re delivering and where — but new technologies are changing those trade-offs. Fastly chose to have fewer, more powerful, located POPs, but Berman notes that “the shift to more edge compute and running WebAssembly is certainly tilting the ratios of things like CPU to memory to disk needed at the edge.”
AI at the edge can deliver better experiences.
Akamai and Cloudflare are both starting to offer more powerful hardware at the edge, including GPUs for running AI inference. “Imagine serverless compute, storage, all the security features, and then combining in the ability to run open source LLM models right there on the edge,” Cloudflare’s Engates suggests.
AI at the edge can deliver better experiences, Akamai’s Alexander agrees. “Whether it’s a commerce application trying to improve conversions, or a financial services company trying to detect fraud, I think more data available at the right time will really help with some of those outcomes.”
Dedicated NPUs and technologies like ONNX will help here, reducing inferencing costs and keeping power demands down (which should be reflected in CDN charges).
Don’t Forget To Turn It On
CDNs only work if you use them; it might sound obvious, but developers often struggle with the complexity of setting up CDNs and the logic of what to include, because they’re now working with a multitier distributed system that might span multiple clouds and providers rather than a simple app or web site, with traffic flowing between all of those rather than just from the app or website to the end user. Routing traffic doesn’t happen automatically: It needs to be set up — and also maintained as all the providers you depend on make changes to their own infrastructure.
Forgetting to route microservices and APIs as well as web content through the CDN — something even large services get wrong from time to time — can cause performance issues when usage ramps up some time after launch. Part of the problem is that the people writing code are often many steps removed from the team that handles those network settings, and may not have a deep understanding of their infrastructure or the dependencies of the services and frameworks they’re working with.
CDNs aren’t “set and forget.” Even if it works well initially, infrastructure and services change.
“How many of them are actually thinking, every time they spin up a new microservice, that if they don’t put it under the host name being routed by a CDN, it’s going to be outside the CDN,” asks Upadhyaya from APIContext. “Sometimes you end up with applications and mobile apps where maybe the ‘front of house’ is over a CDN: fully distributed, terminating at the edge, great last-mile delivery. But then all the core microservices behind it are routing to a single piece of tin in Virginia because the developer created a new domain and forgot to put it underneath the thing that’s configured by the network team.”
CDNs aren’t “set and forget.” Even if it works well initially, infrastructure and services change. “We see people push out applications where they do the testing, everything is great, but six months later things have started to slow down, they’re not optimized and they come back and [discover] a lot of refactoring needs to happen,” Alexander said.
Not using your CDN fully results in missed opportunities, including security issues as well as performance problems, he warns.
“With choice and flexibility, comes risk. You can build an application very easily but often without fully understanding all the potential issues. We’ve seen developers launch an application and leave an API endpoint unsecured that they didn’t know explicitly was going to be internet-facing.”
“The great thing about CDNs is, one, they’re inline, and secondly, you can offload,” Upadhyaya adds. “There’s so much done back at source where it could be done in the CDN; token refreshing, even schema validation could be done on the fly. There’s so much out there that can be workhorses for them; if they could just offload this stuff, there’s a lot of hygiene and guardrails they’d get out of the box.”
“We’re asking a lot of developers today, because they’ve got to be network experts, cyber experts, database experts, frontend developers — and get their job done.”
– Mayur Upadhyaya, APIContext CEO
That’s another reason the edge has become so important for CDNs. “Even if you’re only targeting a small group of users or a small geographic area, your attack surface is global; you can get DDoSed and attacked from around the world,” Fastly’s Bergman points out. “You protect your more expensive origin from attacks around the world. Just as the cheapest place to serve a piece of content is as close to the user as possible, the cheapest place to stop an attack from consuming resources is as close to the attacker as possible.”
Organizations are increasingly buying in services for critical areas like identity and access management rather than writing their own. They’re probably better written and more secure, but the chain of dependencies they bring with them means your application now depends on a complex interconnected network that could involve telecoms infrastructure, cloud platforms, multiple services and your own on-premises systems.
“You have your own digital delivery chain that’s dependent on the public cloud, the public internet, your own internal APIs and all the complexities about how your developers might stick them together, with or without CDNs, and all the third-party systems you buy have this same amount of fragility in them and the same challenges,” Upadhyaya warns.
All the capabilities of modern CDNs are a big opportunity for how you deliver apps with a great user experience — but also a lot of work. “We’re asking a lot of developers today, because they’ve got to be network experts, cyber experts, database experts, frontend developers — and get their job done.”
The post The Modern CDN Means Complex Decisions for Developers appeared first on The New Stack.
Comments are closed.