Acquire all necessary knowledge on Azure Storage Account in this definite guide. In a nutshell, it covers storage services, comparison on the core storage services, properties and metrics of Storage Accounts, tools available to manage and monitor Azure Storage Account, and much more. This guide also justifies how Serverless360 turns out to be the best tool to manage and monitor the Azure Storage Account.
Each organization will have specific requirements for its cloud-hosted data. It can either store data within a particular region or need separate billing for different data categories. Azure storage accounts help to solemnize these types of policies and apply them to the Azure data. Azure offers enormous ways to store the data. There are several database options like Azure SQL Database, Azure Cosmos DB, and Azure Table Storage. Azure provides various ways to store and send messages, such as Azure Queues and Event Hubs. Even loose files can be stored using services like Azure Files and Azure Blobs.
A storage account is a container that bands a set of Azure Storage services together. Only data services from Azure Storage can be comprised in a storage account. Integrating data services into a storage account allows the user to manage them as a group. The settings specified while creating the account, or setting that is changed after creation, is applicable everywhere. Once the storage account gets deleted, all the data stored inside gets removed.
Let us consider a machine manufacturing industry that produces machinery and instruments such as pulley, screws, and wedges. The products get marketed to stores which then sell them to consumers.
The designs and manufacturing processes of the industry get preserved as trade secrets. The spreadsheets, documents, and instructional videos that record this information are significant to the business and necessitate redundant storage. This data gets primarily retrieved from the main factory, so the user would prefer to store it in a nearby datacentre. The expenditure for this storage should be billed to the manufacturing department.
The industry also owns a sales group that creates demonstration and advertisement videos to promote consumers’ products. Priority for this data is a low cost, rather than redundancy or location. This storage should be billed to the sales team. Such business handles require multiple Azure storage accounts, and each storage account will incorporate the appropriate settings for the data it holds.
Azure Storage provides different types of storage accounts. Each type supports unique features and has its pricing model. Consider these differences before creating a storage account to work out the best account for the applications. The types of storage accounts are:
Core storage services provide an enormously scalable object store for data objects, disk storage for Azure virtual machines (VMs), a file system service for the cloud, a messaging store for reliable messaging, and a NoSQL store.
The Azure Storage platform comprises the following data services:
Azure Blob storage is an object storage solution designed for the cloud. Blob storage gets augmented for storing a massive amount of unstructured data. Unstructured data is data that does not stick to a specific data model or definition, like text or binary data. Blob storage objects can be accessed by the user or client application via HTTP/HTTPS from any part of the world. Azure Storage Rest API, Azure PowerShell, Azure CLI, or an Azure Storage client library is used to access Blob Storage objects. The flow and resources of a Blob Storage is pictured below,
The Blob Storage consists of a storage account with a container residing in it. Containers hold the respective blobs in them.
Azure Storage supports three types of blobs; they are:
Block Blobs
Block blobs are designed to store text and binary data. Block blobs are built with blocks of data that can be managed independently. It has a storage capacity of about 4.75 TiB of data. Larger block blobs are currently in preview and have storage up to 190.7 TiB
Append Blobs
Append Blobs are created with blocks like block blobs but are enhanced for append operations. Append blobs are preferred for scenarios like logging data from virtual machines.
Page Blobs
Page Blobs store random access files ranging up to 8 TB in size: these blobs stock virtual hard drive (VHD) files and function as disks for Azure virtual machines.
Let us consider an amplified-reality gaming company. The game runs on every mobile platform without any criteria. The scenario here is to add a new feature that allows users to record video clips of their gameplay and further upload them to the servers. Users can either watch the clips directly in-game or through the game website. A log on every upload and viewing can be planned and maintained to benefit analytics and traceability.
The user requirement would be a storage solution that can grip thousands of simultaneous uploads, a huge count of video data, and constantly growing log files. There is also a need for viewing functionality in all the mobile apps and website, so the user requires API access from multiple platforms and languages. Azure Blob storage could be an ideal solution for this application
Azure Blob Storage was designed to serve specific needs. If the business use case needs to store unstructured data like audio, video, images, etc., then you should probably go with this option. The objects which are being stored in Blob does not necessarily have an extension.
The following points describe the use case scenarios:
Azure storage offers various access tiers, which allows storing the blob object data in a very cost-effective manner. The available access tiers include:
The storage access cost for the hot access tier is comparatively higher than the storage costs of cool and archive tiers. But access cost of hot access tiers is low. Real-time usage for the hot access tier include:
In comparison to hot access tiers, the cool access tier has lower storage costs and higher access costs. This tier is envisioned for data that will stay back in the cool tier for at least 30 days. Real-time usage for the cool access tier include:
The storage cost of archive access is low, but the data retrieval cost is high when compared to the hot and cool tiers. Data must sustain in the archive tier for at least 180 days or will be subjected to an early deletion charge. The retrieval time of data in an archive access tier might consume more time depending on the priority of the rehydration. Real-time usage for the Archive Access Tier include:
Data Storage Prices
Blob Storage | PREMIUM | HOT | COOL | ARCHIVE |
---|---|---|---|---|
First 50 terabyte (TB) / month | $0.18 per GB | $0.022 per GB | $0.01 per GB | $0.00099 per GB |
Next 450 TB / Month | $0.18 per GB | $0.0212 per GB | $0.01 per GB | $0.00099 per GB |
Over 500 TB / Month | $0.18 per GB | $0.0203 per GB | $0.01 per GB | $0.00099 per GB |
Operations and Data Transfer Prices
Blob Storage | PREMIUM | HOT | COOL | ARCHIVE |
---|---|---|---|---|
Write operations (per 10,000)1 | $0.021 | $0.05 | $0.10 | $0.11 |
List and Create Container Operations (per 10,000)2 | $0.06 | $0.05 | $0.05 | $0.05 |
Read operations (per 10,000)3
Archive High Priority Read (per 10,000)5 |
$0.0017 | $0.004 | $0.01 | $5.50
$30 |
All other Operations (per 10,000), except Delete, which is free | $0.0017 | $0.004 | $0.004 | $0.004 |
Data Retrieval (per GB)4
Archive High Priority Retrieval (per GB)5 |
Free | Free | $0.01 | $0.022
$0.06 |
Data Write (per GB)4 | Free | Free | $0.0025 | Free |
Checkout Azure Page Blobs Storage Pricing for more details
Azure Files provide fully managed File shares in the cloud that are approachable via the industry-standard SMB. Azure File shares can be attached parallelly by cloud or on-premises deployments of Windows, Linux, and macOS. It can be cached on Windows servers with Azure File Sync for quicker access. It permits the user to set up highly obtainable network file shares that can be accessed by using the standard Server Message Block (SMB) protocol. Multiple VMs can share similar files with both read and write permissions.
The only contrast between Azure Files and files on a corporate file share is, user can access the files from anywhere by using a URL that points to the file and contains a shared access signature (SAS) token. SAS tokens can be generated by the user; they allow specified access to a private asset for a specific time period.
The storage of file shares is one type of data that can be contained in an Azure Storage account. File shares can be implemented for many real-time scenarios:
When it comes to file sharing, the end user should not be allowed to access the copies of the file from its URI and need to be mapped locally in the computers. This is when Azure File Storage fits customer needs. File Storage can be used if the business use case needs to deal mostly with standard File extensions like *.docx, *.png and *.bak, then you should probably go with this storage option.
Let us consider a financial company that is migrating an application to Azure that creates reports and data exports for users and other systems to consume. The company stores these reports and data exports on both NAS devices and Windows file shares. The files are stored this way so they can be easily shared between systems. The company wants to consolidate the storage of these files to a native cloud service. They want to continue to use the Server Message Block (SMB) to access the files securely. The main concern is that they want to reduce the impact on existing applications, systems, and users. The company wants to use a drop-in replacement for their existing SMB protocol shares. They intend that there won’t be any code changes required to support the moved data.
The following are the solutions provided by Azure Files for the use case scenario:
“Lift and shift” applications
The foremost challenge faced by the company is shifting its application to Azure. Azure Files ease the “lift and shift “of applications to the cloud, where the file application or user data is stored in the file share.
Replace or supplement on-premises File servers
After shifting the application to the cloud, the company wants to consolidate the storage of these files to a native cloud service. Azure Files is accustomed to completely replace or supplement any on-premise file servers or NAS devices.
Simplify cloud development
The company’s final requirement is they would like to continue the usage of Server Message Block (SMB) to access its files safely. In Azure Files, user can store development and debugging tools that need to be accessed from many virtual machines.
Azure Blob Storage and File Storage, both services have their own defined properties and are implemented in different scenarios. Azure Files provides fully managed and organized cloud file shares that can be accessed from anywhere. Azure Blob Storage permits the storage of unstructured data and it can be accessed at a massive scale
Consider a development environment where every developer needs access to IDE and tools without using the internet to download it. In this situation, Azure Blob Storage would meet the need and using which the developer can only store development tools then give a link to the team to access the Blob location.
For implementing a File server in an organization, the user should choose the Azure Files option. A File server is used to share Files across departments in an organization. When it comes to File sharing, end user should not be allowed to access the copies of the file from its URI and need to be mapped locally in the computers. This is when Azure File Storage fits the organization’s need.
Below are prices for Data storage and Operations and Data Transferring in Azure File,
Pricing for Data Storage
File Storage | STANDARD |
---|---|
GB/month | $0.06 per used GB |
Snapshots GB/month | $0.06 per used GB |
Pricing for Operations and Data Transferring
File Storage | STANDARD |
---|---|
Put, Create Container Operations (per 10,000) | $0.015 |
List Operations (per 10,000) | $0.015 |
All other operations except Delete, which is free (per 10,000) | $0.0015 |
Geo-Replication Data Transfer (per GiB) | N/A |
Checkout, Azure Files Pricing for more details.
Azure Files provide fully managed File shares in the cloud that are approachable via the industry-standard SMB. Azure File shares can be attached parallelly by cloud or on-premises deployments of Windows, Linux, and macOS. It can be cached on Windows servers with Azure File Sync for quicker access. It permits the user to set up highly obtainable network file shares that can be accessed by using the standard Server Message Block (SMB) protocol. Multiple VMs can share similar files with both read and write permissions.
The only contrast between Azure Files and files on a corporate file share is, user can access the files from anywhere by using a URL that points to the file and contains a shared access signature (SAS) token. SAS tokens can be generated by the user; they allow specified access to a private asset for a specific time period.
The storage of file shares is one type of data that can be contained in an Azure Storage account. File shares can be implemented for many real-time scenarios:
When it comes to file sharing, the end user should not be allowed to access the copies of the file from its URI and need to be mapped locally in the computers. This is when Azure File Storage fits customer needs. File Storage can be used if the business use case needs to deal mostly with standard File extensions like *.docx, *.png and *.bak, then you should probably go with this storage option.
Let us consider a financial company that is migrating an application to Azure that creates reports and data exports for users and other systems to consume. The company stores these reports and data exports on both NAS devices and Windows file shares. The files are stored this way so they can be easily shared between systems. The company wants to consolidate the storage of these files to a native cloud service. They want to continue to use the Server Message Block (SMB) to access the files securely. The main concern is that they want to reduce the impact on existing applications, systems, and users. The company wants to use a drop-in replacement for their existing SMB protocol shares. They intend that there won’t be any code changes required to support the moved data.
The following are the solutions provided by Azure Files for the use case scenario:
“Lift and shift” applications
The foremost challenge faced by the company is shifting its application to Azure. Azure Files ease the “lift and shift “of applications to the cloud, where the file application or user data is stored in the file share.
Replace or supplement on-premises File servers
After shifting the application to the cloud, the company wants to consolidate the storage of these files to a native cloud service. Azure Files is accustomed to completely replace or supplement any on-premise file servers or NAS devices.
Simplify cloud development
The company’s final requirement is they would like to continue the usage of Server Message Block (SMB) to access its files safely. In Azure Files, user can store development and debugging tools that need to be accessed from many virtual machines.
Azure Blob Storage and File Storage, both services have their own defined properties and are implemented in different scenarios. Azure Files provides fully managed and organized cloud file shares that can be accessed from anywhere. Azure Blob Storage permits the storage of unstructured data and it can be accessed at a massive scale
Consider a development environment where every developer needs access to IDE and tools without using the internet to download it. In this situation, Azure Blob Storage would meet the need and using which the developer can only store development tools then give a link to the team to access the Blob location.
For implementing a File server in an organization, the user should choose the Azure Files option. A File server is used to share Files across departments in an organization. When it comes to File sharing, end user should not be allowed to access the copies of the file from its URI and need to be mapped locally in the computers. This is when Azure File Storage fits the organization’s need.
Below are prices for Data storage and Operations and Data Transferring in Azure File,
Pricing for Data Storage
File Storage | STANDARD |
---|---|
GB/month | $0.06 per used GB |
Snapshots GB/month | $0.06 per used GB |
Pricing for Operations and Data Transferring
File Storage | STANDARD |
---|---|
Put, Create Container Operations (per 10,000) | $0.015 |
List Operations (per 10,000) | $0.015 |
All other operations except Delete, which is free (per 10,000) | $0.0015 |
Geo-Replication Data Transfer (per GiB) | N/A |
Checkout, Azure Files Pricing for more details.
Azure Queue storage is an Azure service that implements cloud-based queues. Each queue maintains an inventory of messages. Application components access a queue employing a REST API or an Azure-supplied client library. Typically, you’ll have one or more sender components and one or more receiver components. Sender components add messages to the queue. Messages are retrieved from the front of the queue for processing by receiver components. The subsequent illustration shows multiple sender applications adding messages to the Azure Queue and one receiver application retrieving the messages. Storage Queues are part of the Azure Storage infrastructure, feature a simple REST-based GET/PUT/PEEK interface, providing reliable, persistent messaging within and between services.
URL format:
Queues are addressable using the subsequent URL format:
https://
The following URL addresses a queue within the diagram:
https://myaccount.queue.core.windows.net/images-to-download
Storage account: All access to Azure Storage is completed through a storage account. For information about storage account capacity.
Queue: A queue contains a group of messages. The queue name must be all lowercase. For information on naming queues.
Message: A message, in any form, with range up to 64 KB. Before version 2017-07-29, the utmost time-to-live allowed is seven days. For version 2017-07-29 or later, the utmost time-to-live are often any positive number, or -1 indicating that the message doesn’t expire. If this parameter is neglected, the default time-to-live is seven days.
Imagine the user works as a developer for a major news organization that reports breaking news alerts. The company employs a worldwide network of journalists that are constantly sending updates through a web portal and a mobile app. A middle tier web service layer then takes those alert updates and publishes them online through several channels. However, it’s been noticed the system is missing alerts when globally significant events occur.
The middle tier provides plenty of capacity to handle normal loads. However, a look at the server logs revealed the system was overloaded when several journalists tried to upload larger breaking stories at the same time. Some writers complained the portal became unresponsive, and others said they lost their stories altogether. The user has spotted a direct correlation between the reported issues and the spike in demand on the middle tier servers.
Clearly, user needs a way to handle these unexpected peaks. At such a situation user doesn’t want to add more instances of the website and middle tier web service because they are expensive and, under normal conditions, redundant. They could dynamically spin up instances, but this takes time and they would have the issue waiting for new servers to come online.
This problem can be solved by using Azure Queue storage. A storage queue is a high-performance message buffer that can act as a broker between the front-end components and the middle tier. The front-end components place a message for each new alert into a queue. The middle tier then retrieves these messages one at a time from the queue for processing. At times of high demand, the queue may grow in length, but no stories will be lost, and the application will remain responsive. When demand drops back to normal levels, the web service will catch up by working through the queue backlog.
Below are prices for Data storage and Operations and Data Transferring in Azure Storage Queue,
Pricing for Data Storage
Storage Capacity | LRS | GRS | RA-GRS |
---|---|---|---|
Storage in GB/ Month | $0.045 per GB | $0.06 per GB | $0.0075 per GB |
Pricing for Operations and Data Transferring
LRS | GRS | RA-GRS | |
---|---|---|---|
Queue Class 1* operations (in 10,000) | $0.0004 | $0.0004 | $0.0004 |
Queue Class 2** operations (in 10,000) | $0.0004 | $0.0004 | $0.0004 |
Geo-replication data transfer (per GB) | N/A | Free | Free |
Checkout, Azure Queues Storage Pricing for detailed pricing.
Azure Table storage behaves as a service that stores structured NoSQL data inside the cloud, producing an attribute store with a schema less design. Because Table storage is schema less, it is easy to adapt your data because the needs of your application evolve. Access to Table storage data is fast and cost-effective for several sorts of applications and is usually lower in cost than traditional SQL for similar volumes of knowledge.
Table storage is employed to store flexible datasets like user data for web applications, address books, device information, or other metadata that the service requires. User can store any number of entities in a table, and a storage account may contain any number of tables, up to the capacity limit of the storage account Azure Table Storage Pricing. Azure tables are perfect for storing structured, non-relational data. Real time uses of Table storage include:
Below are prices for Data storage and Operations and Data Transferring in Azure Storage,
Azure charges $0.00036 per 10,000 transactions for tables. Any kind of operation alongside the storage is counted as a transaction, including reads, writes and deletes.
Checkout Azure Tables Storage Pricing for detailed pricing.
Azure managed disks are block-level storage parts that are managed by Azure and used with Azure Virtual Machines. Managed disks are similar a physical disk in an on-premises server but virtualized. In managed disks, all you must do is specify the disk size, type, and provision the disk. Once the disk is provisioned, Azure handles the rest. Each disk can take one of three roles in a virtual machine:
The user manages a healthcare organization, and he is beginning a lift-and-shift migration to the cloud where many of their systems will be running on Azure virtual machines. These systems have a variety of usage and performance profiles which are highly confidential. The user is concerned about the storage and does not want to access that data outside the virtual machine.
To address these needs, the organization’s got to option is Azure Disk Storage. The Azure Disk Storage is capable of,
Azure Managed Disks are the new and recommended disk storage offering to be used with Azure virtual machines for persistent storage of knowledge. Customer can use multiple Managed Disks with each virtual machine. Azure offers four varieties of Managed Disks — Ultra Disk, Premium SSD Managed Disks, Standard SSD Managed Disks, Standard HDD Managed Disks. For Azure Managed Disks, user will be billed on an hourly basis.
Checkout Pricing – Managed Disks for detailed pricing.
Azure Storage accounts provide several high-level security benefits for the data in the cloud:
All data written to Azure Storage is automatically encrypted by Storage Service Encryption (SSE) with a 256-bit Advanced Encryption Standard (AES) cipher. SSE automatically encrypts data when writing it to Azure Storage. When data is read from Azure Storage, Azure Storage decrypts the data before returning it. This process incurs no additional charges and doesn’t degrade performance. It cannot be disabled.
Keep the data secure by enabling transport-level security between Azure and the client. It is advisable to use HTTPS to secure communication over the public internet. When the REST APIs is called to access objects in storage accounts, user can enforce the use of HTTPS by requiring Secure transfer for the storage account. After enabling secure transfer, connections that use HTTP will be refused. This flag will also enforce secure transfer over SMB by requiring SMB 3.0 for all file share mount.
Azure Storage supports cross-domain access through cross-origin resource sharing (CORS). CORS uses HTTP headers so that a web application at one domain can access resources from a server at a different domain. By using CORS, web apps ensure that they load only authorized content from authorized sources. CORS support is an optional flag that can be enabled on Storage accounts.
Azure Storage supports Azure Active Directory and role-based access control (RBAC) for both resource management and data operations. To security principals, user can assign RBAC roles that are scoped to the storage account. Use Active Directory to authorize resource management operations, such as configuration. Active Directory is supported for data operations on Blob and Queue storage.
Auditing is another part of controlling access. User can audit Azure Storage access by using the built-in Storage Analytics service. Storage Analytics logs every operation in real time, and you can search the Storage Analytics logs for specific requests. Filter based on the authentication mechanism, the success of the operation, or the resource that was accessed.
Azure Monitor maximizes the supply and performance of the applications and services by delivering a comprehensive solution for collecting, analyzing, and working on telemetry from cloud and on-premises environments. It helps us understand how the applications are performing and proactively identifies issues affecting them and therefore the resources they depend upon.
Azure Storage Analytics performs logging and offers metrics data for a storage account. User can use this data to trace requests, examine usage trends, and identify issues with the storage account. To use Storage Analytics, user must enable it individually for each service to be monitored. This can be enabled from the Azure portal.
The gathered data is stored in an eminent blob for logging and in renowned tables for metrics, which may be accessed using the Blob service and Table service APIs. Storage Analytics has a 20 TB limit on the quantity of stored data that is independent of total limit for the storage account.
Once the desired Storage Account is configured to the Diagnostics option under the Monitoring section, user will be able to define the type of metrics data to monitor and the retention policy for the data. A default set of metrics is displayed in charts on the Storage Account blade, as well as the individual service blades. Once the metrics are enabled for a service, it may take up to an hour for data to appear in its charts.
For more details on Azure Storage Monitoring, see Monitor a storage account in the Azure portal.
Managing the activities in a service like Azure Storage Account, which deals with different storage services requires a tool with outstanding management and monitoring capabilities. Serverless360 comes into the picture to fulfil all the complex management needs required by the Azure services. Know more about Serverless360 to deal with Azure Storage Account in real-time.
With Serverless360 monitoring abilities user can monitor the Storage Files, Storage Blobs and Storage Queues in a Status or Threshold monitor. Below is the list of properties on which these entities can be monitored.
Entity Type | Properties those can be monitored |
---|---|
Storage Blob | Lease State, Blob Count, Blob Size (Bytes), Block Blob Count, Block Blob Size(Bytes),Page Blob Count, Page Blob Size(Bytes), Append Blob Count, Append Blob Size(Bytes) |
Storage File | Usage (Bytes) |
Storage Queue | Approximate Message Count |
Status Monitor generates a report at specific times in a day representing the state of the entities against the desired values. Threshold Monitor generates a report when certain properties violate desired values for a specified period. Predominantly, Serverless360 monitors the Blobs, Files and Queues based on its properties rather than its metrics, unlike Azure Portal.
Below images depicts the Blob, Queue and Files configuration in the Status and Threshold Monitor of the Serverless360 application.
Azure Blob Configuration,
Azure Storage Queue Configuration,
Azure Files Configuration,
With Serverless360, users can get to view the message content in the Storage Queues by retrieving the messages.
It is also possible to view the properties of the messages in the Storage Queue like dequeue count, Insertion time and more.
Consider there is a need to send the messages in a queue to another queue or to purge the old messages in a queue. With Messages processing in Serverless360, the user can update, resubmit the messages, repair & resubmit the message and Delete the message.
If the number of messages in the Storage Queue exceeds the limit that can be deleted manually, an automated task can be created to purge the Storage Queue periodically when the need is to deal with a huge volume of messages.
Also, explore these documents and webinar sessions for deeper knowledge,
Azure Storage Account Monitoring and Management – Serverless360
Monitoring Storage Accounts – Azure Storage Accounts
Azure Storage documentation – Microsoft Docs
This content will be continuously revamped to stay up to date.