Dynamic Partial-Parallel Data Layout for Efficient Video Surveillance Storage
DPPDL: A Dynamic Partial-Parallel Data
Layout for Green Video Surveillance
Storage
Zhizhuo Sun, Quanxin Zhang, Yuanzhang Li, and Yu-An Tan
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 28, NO. 1, JANUARY 2018
1
Problem
●
Energy consumption of storage in video surveillance increase rapidly.
○
Video acquisition, processing, and analysis on video surveillance require energy-
efficient storage.
●
Common RAID supply surplus performance for write-dominant application
○
Not suitable for processing high-intensity fluctuating workload
●
In video surveillance, uneven workload is common.
○
Video standards (e.g., H.264/MPEG-4 AVC) based on spatial and
temporal redundancy
2
Contribution
●
Dynamic Partial-Parallel Data Layout for video surveillance is proposed.
●
Dynamically providing degree of parallelism according to performance
requirement
●
Be energy efficient on huge capacity condition that occurs in video
surveillance.
●
Different from statically grouping disk or data, DPPDL schedule any number of
disk concurrently to provide appropriate performance.
3
Observation
●
High intensity uneven workload
●
Buffering or data redirection performs
less efficiently than writing dat directly.
●
Redundancy protection is also increase
total cost.
4
Design and Policy
●
Objectives:
1. The degree of parallelism can be dynamically adjusting according to
requirements.
2. Deleting data chronologically without conflicting to 1.
●
E.g.,
Deleted storage space where is 2-disk degree of parallelism (the oldest data)
but the workload is 5-disk degree.
5
Design and Policy
●
Basic data layout
6
Design and Policy
●
Dynamic address mapping of storage space
●
Example workload
●
Macro level: Guarantee deleting the data
obeying FIFO rules
●
Micro level: Stripe perform appropriate
degree of parallelism for uneven workload
Find the max stripe
from current bank
Find stripe from next
bank
Release the space
from next bank
Mapping to disk
address
No
Yes
No
Yes
7
Design and Policy
●
Contention avoidance of disk access
○
make the storage space unable to be
utilized totally.
○
However, they proove the unused ratio
is low even in the worst case.
●
Workload perception
8
Evaluation
●
HD video surveillance system comprising 60 cameras
○
For each camera, avg. transfer rate is 712 KB/s
●
12 disks with 6 TB capacity and 1 disk for parity
●
Linux 3.2.81 kernel. Raid5.c under /driver/md to realize DPPDL
●
Storage server
○
Intel core and memory is 8 GB
○
LSI SAS storage controller on motherboard
●
Peak transfer rate: 1 MB/s; trough transfer rate: 410 KB/s; moderate file
size(240-600 MB)
●
Baseline: PARAID, eRAID, MAID, Hibernator, CacheRAID, and Semi-RAID
9
Evaluation
●
The random write performance is closed to sequential write because DPPDL converts random
into sequential when addressing mapping.
●
Under same degree of parallelism, the read rate of DDPDL is higher because DPPDL mainly
performs small write
10
Evaluation
●
PARAID, eRAID5, and CacheRAID overprovided performance such that consumes extra energy
●
DPPDL has higher write performance than Semi-RAID due to the sequential write after the
address mapping
11
Evaluation
●
PARAID requires enough storage space to save energy, which is difficult in video surveillance.
●
Hibernator has much higher disk migration frequency.
●
MAID require more storage devices
●
CacheRAID is second energy efficient after DPPDL, but higher cost on SSDs.
12
Conslusion
●
DPPDL is proposed for green surveillance storage.
●
Promote energy efficiency while tolerating single disk failure.
●
Meet the performance requirement but require almost fewest storage
devices.
13
Addressing the energy consumption challenge in video surveillance storage, a Dynamic Partial-Parallel Data Layout (DPPDL) is proposed to handle fluctuating workloads efficiently. By dynamically adjusting parallelism and ensuring chronological data deletion, the system aims to optimize performance and energy efficiency. The design also focuses on contention avoidance for improved storage utilization.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
DPPDL: A Dynamic Partial-Parallel Data Layout for Green Video Surveillance Storage Zhizhuo Sun, Quanxin Zhang, Yuanzhang Li, and Yu-An Tan 1 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 28, NO. 1, JANUARY 2018
Problem Energy consumption of storage in video surveillance increase rapidly. Video acquisition, processing, and analysis on video surveillance require energy- efficient storage. Common RAID supply surplus performance for write-dominant application Not suitable for processing high-intensity fluctuating workload In video surveillance, uneven workload is common. Video standards (e.g., H.264/MPEG-4 AVC) based on spatial and temporal redundancy 2
Contribution Dynamic Partial-Parallel Data Layout for video surveillance is proposed. Dynamically providing degree of parallelism according to performance requirement Be energy efficient on huge capacity condition that occurs in video surveillance. Different from statically grouping disk or data, DPPDL schedule any number of disk concurrently to provide appropriate performance. 3
Observation High intensity uneven workload Buffering or data redirection performs less efficiently than writing dat directly. Redundancy protection is also increase total cost. 4
Design and Policy Objectives: 1. The degree of parallelism can be dynamically adjusting according to requirements. 2. Deleting data chronologically without conflicting to 1. E.g., Deleted storage space where is 2-disk degree of parallelism (the oldest data) but the workload is 5-disk degree. 5
Design and Policy Basic data layout 6
Find the max stripe from current bank Design and Policy No Dynamic address mapping of storage space Example workload Find stripe from next bank Yes No Yes A B C D E F G H Workload Release the space from next bank 3 5 2 1 3 2 3 1 degree Macro level: Guarantee deleting the data obeying FIFO rules Micro level: Stripe perform appropriate degree of parallelism for uneven workload Mapping to disk address 7
Design and Policy Contention avoidance of disk access make the storage space unable to be utilized totally. However, they proove the unused ratio is low even in the worst case. Workload perception 8
Evaluation HD video surveillance system comprising 60 cameras For each camera, avg. transfer rate is 712 KB/s 12 disks with 6 TB capacity and 1 disk for parity Linux 3.2.81 kernel. Raid5.c under /driver/md to realize DPPDL Storage server Intel core and memory is 8 GB LSI SAS storage controller on motherboard Peak transfer rate: 1 MB/s; trough transfer rate: 410 KB/s; moderate file size(240-600 MB) Baseline: PARAID, eRAID, MAID, Hibernator, CacheRAID, and Semi-RAID 9
Evaluation The random write performance is closed to sequential write because DPPDL converts random into sequential when addressing mapping. Under same degree of parallelism, the read rate of DDPDL is higher because DPPDL mainly performs small write 10
Evaluation PARAID, eRAID5, and CacheRAID overprovided performance such that consumes extra energy DPPDL has higher write performance than Semi-RAID due to the sequential write after the address mapping 11
Evaluation PARAID requires enough storage space to save energy, which is difficult in video surveillance. Hibernator has much higher disk migration frequency. MAID require more storage devices CacheRAID is second energy efficient after DPPDL, but higher cost on SSDs. 12
Conslusion DPPDL is proposed for green surveillance storage. Promote energy efficiency while tolerating single disk failure. Meet the performance requirement but require almost fewest storage devices. 13
