Skip to content

Commit

Permalink
[New] Using OBJ with CDN guides (#7111)
Browse files Browse the repository at this point in the history
* [New] Using OBJ with CDN guides

* copy edits and formatting changes for both guides

* copy edit

* copy edits and slug change for bucket architecture guide

* Copy edits

* Bigger copy edits

* Copoy edits for Optimizing Object Storage

* Revert "Bigger copy edits"

This reverts commit 54715ea.

* feedback changes and copy edits

---------

Co-authored-by: Nathan Melehan <nmelehan@gmail.com>
  • Loading branch information
jddocs and nmelehan authored Oct 7, 2024
1 parent 432f5b8 commit 6543ff3
Show file tree
Hide file tree
Showing 9 changed files with 197 additions and 0 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
---
slug: optimizing-obj-bucket-architecture-for-akamai-cdn
title: "Optimizing Object Storage Bucket Architecture for Akamai CDN"
description: "This guide discusses design strategies and best practices for optimizing Linode Oject Storage bucket architecture for integrating with Akamai CDN."
authors: ["Akamai"]
contributors: ["Akamai"]
published: 2024-09-27
keywords: ['object storage','cdn','delivery','linode object storage','akamai cdn','akamai connected cloud','bucket architecture']
license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)'
external_resources:
- '[Object Storage Product Documentation](https://techdocs.akamai.com/cloud-computing/docs/object-storage)'
- '[Akamai Content Delivery Documentation](https://techdocs.akamai.com/platform-basics/docs/content-delivery)'
- '[Using Object Storage With Akamai CDN](/docs/guides/using-object-storage-with-akamai-cdn/)'
---

Linode Object Storage can be an efficient, cost-effective solution for streaming and data delivery applications when used as an origin point for Akamai CDN. Since Object Storage is a part of Akamai Connected Cloud and uses the same backbone as Akamai CDN, egress can also be significantly reduced.

Your Object Storage bucket architecture is critical to performance success. In particular, distributing content across multiple buckets helps with load distribution, CDN optimization, and adds security benefits like segmentation and origin obfuscation. This guide walks through bucket design strategies using a commerce site example, including an optimal bucket architecture for Akamai CDN integration.

## How Object Storage Works

[Object Storage](https://techdocs.akamai.com/cloud-computing/docs/object-storage) stores files in a “flat” or unstructured file structure. This means that bucket contents do not exist in a hierarchy like traditional file structures; however, hierarchy can be emulated by creating folders within a bucket. Files (or “objects”) are stored alongside their rich metadata, and access can be granted on a per-object level, with each object assigned a unique URL.

## Bucket Architecture Strategies

### Example Scenario

For the example bucket architecture, consider the following scenario:

- An online commerce site serving audio, video, and image-based content
- All audio, video, and image files are stored on object storage as an origin point

### What "Not" To Do

In the first bucket example, the top-level directory `store` has paths to three lower-level directories containing different types of content: music, video, and images. `store` and its sub-directories are in a single bucket. This bucket has a defined requests per second (RPS) limit, where the RPS limit is shared by all objects in the bucket.

Paths in this bucket are: `store/music`, `store/video`, and `store/images`.

![Single Bucket Arch Level 1](Single-bucket-arch-level1.png)

Additional content such as copyrighted music or free music is placed as sub-directories under the `store/music` directory. Paths in this bucket now include: `store/music`, `store/music/copyright`, `store/music/free`, `store/video`, and `store/images`.

![Single Bucket Arch Level 2](Single-bucket-arch-level2.png)

This architecture places content paths and object endpoints all within the same bucket, `store`. This may potentially result in poor performance with rate limits for the bucket being reached as the number of users and requests increases.

{{< note >}}
For up-to-date technical specifications of Linode Object Storage such as bucket and rate limits, see: [Object Storage: Technical Specifications and Considerations](https://techdocs.akamai.com/cloud-computing/docs/object-storage#technical-specifications-and-considerations)
{{< /note >}}

### What To Do & How To Leverage CDN Advantages

#### CDN Offloading

One way to overcome these issues is by utilizing Akamai CDN and its ability to cache and deliver content closer to end users. Once the CDN caches an object like a song, video, or image, it doesn’t need to be pulled from origin storage again until the object file changes or the cache timeout expires. For example, by offloading requests to the CDN, a single MP3 audio file downloaded 1 million times only requires a single request to the source bucket.

#### Distributed Bucket Architecture

While Akamai CDN can help get around some limitations of a single-bucket architecture, you can improve and optimize your object storage performance even further by **distributing content across multiple buckets**. Doing this results in:

- Overall load distribution
- Increased requests-per-second (RPS) capacity
- Segmentation of content across endpoints can act as a security measure in case of compromise
- Distribution of the number of single endpoint requests from the CDN

Rather than placing music, video, and image content folders within the same top-level directory, you can triple [RPS capacity](https://techdocs.akamai.com/cloud-computing/docs/object-storage#technical-specifications-and-considerations) by placing each type of content (music, video, and images) in their own buckets: `store-music`, `store-video`, and `store-images`. In the example below, additional content is placed underneath the top-level directory of each bucket:

- Paths for the `store-music` bucket include: `store-music`, `store-music/copyright`, and `store-music/free`
- Paths for the `store-video` bucket include: `store-video`
- Paths for the `store-images` bucket include: `store-images`

![Multi-Bucket Arch 1](Multi-bucket-arch1.png)

As the site scales, you can create additional buckets and move sub-category content (such as copyrighted or free music) to their own buckets. This increases RPS capacity by further distributing the number of endpoints from which the CDN can cache content. In the example below, copyrighted music is stored in a new `copyright` bucket. RPS capacity is now four times the original bucket architecture:

- Paths for the `copyright` bucket include: `copyright`
- Paths for the `store-music` bucket now include: `store-music` and `store-music/free`
- Paths for the `store-video` bucket include: `store-video`
- Paths for the `store-images` bucket include: `store-images`

![Multi-Bucket Arch 2](Multi-bucket-arch2.png)

## CDN & the Relationship To Bucket Design

### CDN Considerations

Each bucket in your architecture has the ability to serve as a single origin endpoint from which Akamai CDN can pull content. This results in a distributed backend with less opportunities for RPS limits to be reached. The more distributed your backend architecture is, the higher the capacity for scaling.

### Relationship To Bucket Design

CDNs can often overcome flaws of poorly architected environments. However, when a bucket architecture is designed well, the benefits can directly translate to the CDN. Object Storage bucket architecture should be designed to be functional, scalable, performant, resilient, and cost efficient so that Akamai CDN serves your content as effectively as possible.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
---
slug: using-object-storage-with-akamai-cdn
title: "Using Object Storage With Akamai CDN"
description: "This guide presents the benefits and strategies for pairing Linode Object Storage with Akamai CDN for data delivery."
authors: ["Akamai"]
contributors: ["Akamai"]
published: 2024-09-27
keywords: ['object storage','cdn','delivery','linode object storage','akamai cdn','akamai connected cloud']
license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)'
external_resources:
- '[Object Storage Product Documentation](https://techdocs.akamai.com/cloud-computing/docs/object-storage)'
- '[Akamai Content Delivery Documentation](https://techdocs.akamai.com/platform-basics/docs/content-delivery)'
---

Object storage stores data in an unstructured, flat format where data is stored as “objects”. Object storage is cheaper than block storage, and it can be especially useful for large volumes of unstructured data like photos, videos, documents, web pages, sensor data, and audio files. Cloud-based object storage systems distribute this unstructured data across multiple physical devices for redundancy, while users are able to access the content from a single interface. Object storage is HTTP accessible and ideal for building cloud-native applications that require scale and flexibility, as well as storage for application artifacts like analytics, backups, and archives.

With its reliability and cost-effectiveness, Linode Object Storage can be coupled with Akamai’s content delivery network (CDN) to create a viable solution for applications that need to serve large files and large amounts of data. This includes video on demand (VOD) streaming, ecommerce, firmware update distribution, media-based website content, and other applications.

The solution presented in this guide focuses on the architectural best practices for designing and operating reliable, secure, efficient, and sustainable content storage and delivery systems using [Linode Object Storage](https://techdocs.akamai.com/cloud-computing/docs/object-storage) on Akamai Connected Cloud with [Akamai CDN](https://techdocs.akamai.com/platform-basics/docs/content-delivery).

## Object Storage For Data Delivery

With the right [bucket architecture](/docs/guides/optimizing-obj-bucket-architecture-for-akamai-cdn/), object storage can be used to house content for effective unstructured data delivery. Object storage supports critical features such as encryption, compression, deduplication, and versioning.

Its accessibility via HTTP protocols offers direct access to objects. Object storage doesn't require an intermediary proxy to serve files (i.e. Block Storage Volumes attached to a Compute Instance) and can be accessed by a wide range of APIs.

For example, these factors make object storage ideal for storing unstructured data like video and audio files that don’t require frequent updating and services that require efficient input/output.

## Object Storage Specifications, Considerations, and Strategies for Streaming Data

### Technical Specifications

When architecting applications that source data from object storage -- like streaming video or audio -- consider these technical factors:

- **Bandwidth and latency:** Object storage solutions are not always optimized for high-bandwidth streaming applications. This can lead to potential buffering issues and slower delivery speeds.

- **Scalability:** To preserve infrastructure integrity, object storage systems may have limitations in place for concurrent connections and simultaneous user streams. This can impact the ability to efficiently handle spikes in demand.

- **Retrieval speed:** Without CDN edge-based caching, content streamed directly from object storage origin servers may result in slower data retrieval speeds and higher latency.

- **Data transfer and egress costs:** Streaming directly from object storage can incur high egress costs, because each user request may result in data transfer fees. When using a CDN, only requests to update the CDN cache may incur data transfer fees.

- **Resource limits:** Object storage systems may have limitations on the size of individual objects, potentially restricting the size of videos that can be efficiently streamed.

- **Rate limits:** Object storage solutions often have rate limits imposed on a requests per second (RPS) basis. If users are accessing Object Storage endpoints directly without the use of a CDN, there is a greater chance RPS limits are reached as more users access content.

- **Updating manifests:** Bucket contents can be determined by syncing with a manifest. This means that when source manifests are updated often, buckets may constantly have their contents changed.

### Network Considerations

There are multiple internet-related factors to consider when building out a content delivery solution:

- **First mile:** The content provider connection point. Should content providers set up content to be accessible from a single physical location (i.e. an origin data center), user access is constrained by first mile connectivity limitations.

- **Last mile:** The end-user connection point. Slow user internet connection speeds are uncontrollable and unpredictable. They can also mask or obfuscate problems related to the first mile, network peering, and bottleneck issues.

- **Peering points:** The connection points between two networks. These connections are not guaranteed and are difficult to troubleshoot. Networks benefit financially from connectivity with content providers and end users but not each other.

- **Backbone:** The infrastructure and physical connections that make up the internet. Hardware and network capacity limitations can cause performance issues.

### Data Center-Based Architectures

Consider the data center-based architecture below. Without the use of a CDN, object storage content distribution is limited to individual, origin data centers. Data center-based delivery relies heavily on public networks to pull and deliver content from regional origin points. At scale, this may result in bandwidth issues, high latency, and inefficient experiences for large numbers of users.

![DC-Based Architecture](DC-Based-Architecture.png)

## Advantages of Using a CDN

CDNs are crucial for data delivery solutions because they cache content like images, videos, and other files on edge servers closer to end-users. This reduces latency and overall load on origin servers, improves delivery speed and efficiency, and saves money by reducing origin requests and lowering egress costs. For example, in use cases such as content streaming, CDNs can cache segments from streams to reduce startup times, limit stream interruptions, and eliminate potential buffering issues.

Object storage can’t scale linearly with the number of users, but using it as an origin point for CDN delivery overcomes this by allowing users to interact with the CDN instead of the object storage system. Applications can then serve objects independently of user traffic, with the added benefit of CDN-based security.

### CDN-Based Distributed Architecture

In the CDN-based architectures below, Akamai CDN lives between end users and the origin data point. As end users request information, Akamai CDN sources and caches data from Linode Object Storage on Akamai Connected Cloud, and then serves that data from an edge server close to the end user.

In this scenario, latency is reduced, users never need to interact with the origin, and the object storage infrastructure remains protected and optimized for efficiency and growth.

![CDN-Based Architecture](CDN-Based-Architecture.png)

![Edge vs. DC Latency](Edge-vs-DC-latency.png)

## Benefits of Using Akamai CDN with Object Storage for Data Delivery

Akamai Connected Cloud brings content closer to end-users and offers several technical benefits for streaming applications, including:

- Audience and end-user geographic reach
- Programmability through well-defined APIs
- Flexibility to support structured and semi-structured data
- Scalable consumption models
- Efficient data recovery and retrieval based on detailed metadata
- Capacity advantages via distribution across multiple nodes
- Enhanced data durability and availability through replication and distribution mechanisms
- Limiting reliance on transit networks
- Origin offloading, content caching, and reduced latency
- Significantly reduced object storage-based egress costs since no egress is imposed between Linode Object Storage and Akamai CDN

Object storage is a standardized, key component in many content production tools, enabling seamless integration and organization of large amounts of files. Its interoperability across various applications, tools, and workflows ensures data is centralized and easily accessible. Object storage simplifies content distribution and backup processes, and it provides a secure, cost-effective method for storing and sharing large volumes of unstructured data.

When architected properly and paired with Akamai’s CDN, Linode Object Storage allows for fast, efficient access to vast libraries of content connected on the same backbone. This results in readily available content - a crucial factor for streaming and other data delivery workflows.

## Next Steps

To learn more about object storage bucket design for streaming and steps for CDN integration, see the following guide:

- [Optimizing Object Storage Bucket Architecture for Akamai CDN](/docs/guides/optimizing-obj-bucket-architecture-for-akamai-cdn/)

0 comments on commit 6543ff3

Please sign in to comment.