In this paper, we propose a simply TCP-friendly source adaptation framework to provide a proportional bandwidth sharing service. Our scheme is based on the framework of Monotonic Response Function (MRF), can be used to bound the sending rate of a source within a predefined interval and provide very smooth traffic, and is suitable for multimedia applications over the Internet. As our scheme is also very simple and TCP-friendly, it is easy to be deployed over the current Internet. We verify our scheme with experimental studies.

In this paper we introduce a framework for using quality as an incentive to promote proper application level congestion control. Through integrating a joint-source channel coder and feedback-based congestion control scheme, we are able to construct accurate and efficient quality incentives. The framework is applicable in all network architectures where end-to-end congestion control may be used, and is as such not specific to either best-effort or traffic class-based architectures. The concept is presented along with preliminary simulations that highlight the resulting rate control accuracy. We also discuss how to implement some well-known congestion control schemes within our framework.

The limitation of processing power, battery life and memory capacity of portable terminals requires reducing encoding complexity in mobile communications. Motion estimation (ME) is the most computationally intensive module in a typical video codec, which determines not only the encoder’s performance but also the reconstructed video quality. In this paper, a fast ME algorithm for H.264/AVC baseline profile coding is proposed based on the analysis of motion vector field and error surface, and the statistical distributions of different type macroblocks (MBs). Simulation results showed that: in comparison with MVFAST, the proposed algorithm can decrease the computational load over 7.2% with no requirement of expanding memory capacity while maintaining the same video quality as MVFAST. Furthermore, its simplicity makes it easy to be implemented on hardware.

In this work, we present an evaluation of the performance and error robustness of RTP-based broadcast streaming of high-quality high-definition (HD) H.264/AVC video. Using a fully controlled IP test bed (Hillestad et al., 2005), we broadcast high-definition video over RTP/UDP, and use an IP network emulator to introduce a varying amount of randomly distributed packet loss. A high-performance network interface monitoring card is used to capture the video packets into a trace file. Purpose-built software parses the trace file, analyzes the RTP stream and assembles the correctly received NAL units into an H.264/AVC Annex B byte stream file, which is subsequently decoded by JVT JM 10.1 reference software. The proposed measurement setup is a novel, practical and intuitive approach to perform error resilience testing of real-world H.264/AVC broadcast applications. Through a series of experiments, we evaluate some of the error resilience features of the H.264/AVC standard, and see how they perform at packet loss rates from 0.01% to 5%. The results confirmed that an appropriate slice partitioning scheme is essential to have a graceful degradation in received quality in the case of packet loss. While flexible macroblock ordering reduces the compression efficiency about 1 dB for our test material, reconstructed video quality is improved for loss rates above 0.25%.

Classical unequal erasure protection schemes split data to be protected into classes which are encoded independently. The unequal protection scheme presented in this paper is based on an erasure code which encodes all the data together according to the existing dependencies. A simple algorithm generates dynamically the generator matrix of the erasure code according to the packets streams structure, i.e., the dependencies between the packets, and the rate of the code. This proposed erasure code was applied to a packetized MPEG4 stream transmitted over a packet erasure channel and compared with other classical protection schemes in terms of PSNR and MOS. It is shown that the proposed code allows keeping a high video quality-level in a larger packet loss rate range than the other protection schemes.

Although H.264 video coding standard provides several error resilience tools, the damage caused by error propagation may still be tremendous. This work is aimed at developing a robust and standard-compliant error resilient coding scheme for H.264 and uses techniques of mode decision, data hiding, and error concealment to reduce the damage from error propagation. This paper proposes a system with two error resilience techniques that can improve the robustness of H.264 in noisy channels. The first technique is Nearest Neighbor motion compensated Error Concealment (NNEC) that chooses the nearest neighbors in the reference frames for error concealment. The second technique is Distortion Estimated Mode Decision (DEMD) that selects an optimal mode based on stochastically distorted frames. Observed simulation results showed that the rate-distortion performances of the proposed algorithms are better than those of the compared algorithms.

In this paper, we present a spatio-temporal post-processing error concealment (EC) algorithm designed initially for a H.264 video-streaming scheme over packet-lossy networks. It aims at optimizing the subjective quality of the restored video under the constraints of low delay and computational complexity, which are critical to real-time applications and portable devices having limited resources. Specifically, it takes into consideration the physical property of motion field in order to achieve more meaningful perceptual video quality, in addition to the improved objective PSNR. Further, a simple bilinear spatial interpolation approach is combined with the improved boundary-match (B-M) based temporal EC approach according to texture and motion activity analysis. Finally, we propose a low complexity temporal EC method based on motion vector interpolation as a replacement of the B-M based approach in the scheme under low-computation requirement, or as a complement to further improve the scheme’s performance in applications having enough computation resources. Extensive experiments demonstrated that the proposal features not only better reconstruction, objectively and subjectively, than JM benchmark, but also robustness to different video sequences.

Error-resilient video communication over lossy packet networks is often designed and operated based on models for the effect of losses on the reconstructed video quality. This paper analyzes the channel distortion for video over lossy packet networks and proposes a new model that, compared to previous models, more accurately estimates the expected mean-squared error distortion for different packet loss patterns by accounting for inter-frame error propagation and the correlation between error frames. The accuracy of the proposed model is validated with JVT/H.264 encoded standard test sequences and previous frame concealment, where the proposed model provides an obvious accuracy gain over previous models.

Audio Video coding Standard (AVS) is the latest audio and video coding standard of China. AVS Part 7 (also known as AVS-M) targets mobility applications where error concealment is of great importance. This paper first briefly introduces the general concept of error concealment. Then two error concealment schemes are proposed and implemented on AVS-M decoder under different test conditions. Simulation results of the schemes and suggestions on how to use these tools are also provided.

Reversible variable length codes (RVLCs) have received much attention due to their excellent error resilient capabilities. In this paper, a novel construction algorithm for symmetrical RVLC is proposed which is independent of the Huffman code. The proposed algorithm’s codeword assignment is only based on symbol occurrence probability. It has many advantages over symmetrical construction algorithms available for easy realization and better code performance. In addition, the proposed algorithm simplifies the codeword selection mechanism dramatically.

Dynamic adaptation of multimedia content is seen as an important feature of next generation networks and pervasive systems enabling terminals and applications to adapt to changes in e.g. context, access network, and available Quality-of-Service (QoS) due to mobility of users, devices or sessions. We present the architecture of a multimedia stream adaptation service which enables communication between terminals having heterogeneous hardware and software capabilities and served by heterogeneous networks. The service runs on special content adaptation nodes which can be placed at any location within the network. The flexible structure of our architecture allows using a variety of different adaptation engines. A generic transcoding engine is used to change the codec of streams. An MPEG-21 Digital Item Adaptation (DIA) based transformation engine allows adjusting the data rate of scalable media streams. An intelligent decision-taking engine implements adaptive flow control which takes into account current network QoS parameters and congestion information. Measurements demonstrate the quality gains achieved through adaptive congestion control mechanisms under conditions typical for a heterogeneous network.

A fast mode decision algorithm is proposed in this paper to accelerate the process of transcoding videos into H.264 with arbitrary rate spatial resolution down-scaling. The proposed algorithm consists of three steps. First, an early-stop technique is introduced to determine the 16×16-mode blocks, which take up about 70% of all the macroblocks; then, a bottom-up merging process is performed to determine the mode of rest non-early-stopped blocks; and then, we adopt half-pixel motion estimation to further refine the acquired predictive motion vectors. In order to obtain the predictive motion vectors for early-stop and merging processes, we propose a motion vector composition scheme, which can reuse the information in the input pre-encoded videos to handle the spatial resolution down-scaling. Experimental results showed that our algorithm is about four times faster than the Cascaded-Decoder-Encoder method and has negligible PSNR drop and little bit rate increase.

H.264/MPEG-4 AVC standard appears highly competitive due to its high efficiency, flexibility and error resilience. In order to maintain universal multimedia access, statistical multiplexing, or adaptive video content delivery, etc., it induces an immense demand for converting a large volume of existing multimedia content from other formats into the H.264/AVC format and vice versa. In this work, we study the remultiplexing and resynchronization issue within system coding after transcoding, aiming to sustain the management and time information destroyed in transcoding and enable synchronized decoding of decoder buffers over a wide range of retrieval or receipt conditions. Given the common intention of multiplexing and synchronization mechanism in system coding of different standards, this paper takes the most widely used MPEG-2 transport stream (TS) as an example, and presents a software system and the key technologies to solve the time stamp mapping and relevant buffer management. The solution reuses previous information contained in the input streams to remultiplex and resynchronize the output information with the regulatory coding and composition structure. Experimental results showed that our solutions efficiently preserve the performance in multimedia presentation.

Low energy consumption is one of the main challenges for wireless video transmission on battery limited devices. The energy invested at the lower layers of the protocol stack involved in data communication, such as link and physical layer, represent an important part of the total energy consumption. This communication energy highly depends on the channel conditions and on the transmission data rate. Traditionally, video coding is unaware of varying channel conditions. In this paper, we propose a cross-layer approach in which the rate control mechanism of the video codec becomes channel-aware and steers the instantaneous output rate according to the channel conditions to reduce the communication energy. Our results show that energy savings of up to 30% can be obtained with a reduction of barely 0.1 dB on the average video quality. The impact of feedback delays is shown to be small. In addition, this adaptive mechanism has low complexity, which makes it suitable for real-time applications.

A new all-zero block determination rule was used to reduce the complexity of the AVS-M encoder. It reuses the sum of absolute difference of 4×4 block obtained from motion estimation or intra prediction as parameters so that the determination threshold need to be computed only once when quantization parameter (QP) is invariable for given video sequence. This method avoids a lot of computation for transform, quantization, inverse transform, inverse quantization and block reconstruction. Simulation results showed that it can save about 20%~50% computation without any video quality degradation.

A new no-reference blocking artifact metric for B-DCT compression video is presented in this paper. We first present a new definition of blocking artifact and a new method for measuring perceptive blocking artifact based on HVS taking into account the luminance masking and activity masking characteristic. Then, we propose a new concept of blocking artifact cluster and the algorithm for clustering blocking artifacts. Considering eye movement and fixation, we select several clusters with most serious blocking artifacts and utilize the average of their blocking artifacts to assess the total blocking artifact of B-DCT reconstructed video. Experimental results illustrating the performance of the proposed method are presented and evaluated.

With advanced prediction modes of intra prediction, intra coding of H.264/AVC offers significant coding gains compared with previous video coding standards. It uses an important tool called Lagrangian rate-distortion optimization (RDO) technique to decide the best coding mode for a block, but the computational burden is extremely high. In this paper, we proposed an improved fast intra prediction algorithm including block type selection and mode decision algorithm based on analysis of edge feature of a block. Our algorithm filters out unlikely block type and candidate modes to reduce the RDO calculations. Experimental results showed that the proposed algorithm can reduce the computation complexity of intra prediction from 52.90% to 56.31%, with 0.04 dB PSNR degradation and 2% increase of bit rate.

In this paper we describe how progressive download and adaptive streaming can be combined into a simple and efficient streaming framework. Based on the MPEG-4 file format (MP4) we use HTTP for transport and argue that these two components are sufficient for specifying an open streaming architecture. The client selects appropriate chunks from the MP4 file to be transferred based on (1) the header information (i.e. the “moov” box) in the first part of the file and (2) his observation of network throughput. The framework is completely client driven which allows for better server scalability and reduces signalling overhead. We discuss architecture and implementation issues such as complexity, interoperability and scalability and compare to 3GPP PSS Rel-6 adaptive streaming when appropriate. Measurements from a simple MP4/HTTP streaming client are presented showing that appropriate chunks are selected such that increased reliability is achieved.

The effect of receiver buffer size on perceived video quality of an Internet video streamer application was examined in this work. Several network conditions and several versions of the application are used to gain understanding of the response to varying buffer sizes. Among these conditions local area versus wide area, bandwidth estimation based versus non-bandwidth estimation based cases are examined in detail. A total of 1000 min of video is streamed over Internet and statistics are collected. It was observed that when bandwidth estimation is possible, choosing larger buffer size for higher available bandwidth yields quality increase in perceived video.

In this paper, we propose a novel optimal quality adaptation algorithm for MPEG-4 fine granular scalability (FGS) stream over wired network. Our algorithm can maximize perceptual video quality by minimizing video quality variation and increasing available bandwidth usage rate. Under the condition that the whole bandwidth evolution is known, we design an optimal algorithm to select layer. When the knowledge of future bandwidth is not available, we also develop an online algorithm based on the optimal algorithm. Simulation showed that both optimal algorithm and online algorithm can offer smoothed video quality evolution.

Peer-to-peer (P2P) systems are now very popular. Current P2P systems are broadly of two kinds, structured and unstructured. The tree structured P2P systems used technologies such as distributed hash tables (DHT) and hierarchical clustering can search the required target quickly, however, in a tree, the internal node has a higher load and its leave or crash often causes a large population of its offspring’s problems, so that in the highly dynamic Internet environment the tree structure may still suffer frequent breaks. On the other hand, most widely used unstructured P2P networks rely on central directory servers or massive message flooding, clearly not scalable. So, we consider both of the above systems’ advantages and disadvantages and realize that in the P2P systems one node may fail easily, but that when a number of nodes organized as a set, which we call “super node”, the set is robust. Super nodes can be created and updated aware of topology-aware, and used with simple protocol such as flooding or “servers” to exchange information. Furthermore the entire robust super node can be organized into exquisite tree structure. By using this overlay network architecture, P2P systems are robust, efficient, scalable and secure. The simulation results demonstrated that our architecture greatly reduces the alteration time of the structure while decreasing the average delay time, compared to the common tree structure.

Video compression technologies are essential in video streaming application because they could save a great amount of network resources. However compressed videos are also extremely sensitive to packet loss which is inevitable in today’s best effort IP network. Therefore we think accurate evaluation of packet loss impairment on compressed video is very important. In this work, we develop an analytic model to describe these impairments without the reference of the original video (NR) and propose an impairment metric based on the model, which takes into account both impairment length and impairment strength. To evaluate an impaired frame or video, we design a detection and evaluation algorithm (DE algorithm) to compute the above metric value. The DE algorithm has low computational complexity and is currently being implemented in the real-time monitoring module of our HDTV over IP system. The impairment metric and DE algorithm could also be used in adaptive system or be used to compare different error concealment strategies.

The 3rd generation partnership project (3GPP) has defined the protocols and codecs for implementing media streaming services over packet-switched 3G mobile networks. The specification is based on IETF RFCs on audio/video transport. It also adds new features to achieve better adaptation to the mobile network environment. In this paper, we propose an algorithm for handover detection and fast buffer refill that is based on the existing feedback and signaling mechanisms. The proposed algorithm refills the receiver buffer at a faster pace during a limited time frame after a hard handover is detected in order to achieve higher video quality.

We propose an on-demand multipath routing algorithm in a mobile ad hoc network for video transmission and evaluate its real world performance in video streaming application. There have been a number of multipath routing protocols extended from AODV which is a well-known single path routing protocol. Multipath routing protocols indicate good performance in the reduction of route discovery latency and unnecessary routing packets in simulations. We show that the route establishment using source route lists provided by us (Hu and Johnson, 2002) can reduce the route discovery latency, select stable routes automatically, and work well for live video streaming without limitation of the hop count based approaches. We evaluate this proposed method compared with the original AODV by using eight laptop PCs and demonstrate live streaming experiments.

This paper introduces a video application-aware cross-layer framework for joint performance-energy optimization, considering the scenario of multiple users upstreaming real-time Motion JPEG2000 video streams to the access point of a WiFi wireless local area network and extends the PHY-MAC run-time cross-layer scheduling strategy that we introduced in (Mangharam et al., 2005; Pollin et al., in press) to also consider congested network situations where video packets have to be dropped. We show that an optimal solution at PHY-MAC level can be highly suboptimal at application level, and then show that making the cross-layer framework application-aware through a prioritized dropping policy capitalizing on the inherent scalability of Motion JPEG2000 video streams leads to drastic average video quality improvements and inter-user quality variation reductions of as much as 10 dB PSNR, without affecting the overall energy consumption requirements.

In network with a shared channel in TDMA mechanism, it is a core issue to effectively allocate channel time to provide service guarantees for flows with QoS requirements. This paper proposes a simple and efficient time allocation scheme called MES-ESRPT (MCTA at the End of Superframe-Enhanced Shortest Remaining Processing Time) for delay-sensitive VBR traffic in accordance with IEEE 802.15.3 standard. In this algorithm, PNC (piconet coordinator) allocates one MCTA (Management Channel Time Allocation) for each stream which is the process of communication at the end of superframe. During the MCTA period, each transmitter should report current fragments number of the first MSDU (MAC Service Data Unit) and the fragments number of the remainder MSDUs to PNC. In the next superframe, PNC firstly allocates part CTAs (Channel Time Allocation) for each stream based on the remainder fragments number of the first MSDU by SRPT rule, then allocates remainder CTAs for each stream based on all fragments number of remainder MSDUs by the same SRPT rule. Simulation results showed that our proposed MES-ESRPT method achieves significantly better performance in QoS for multimedia streams compared to the existing schemes.