Rajiv Ratn Shah received his B.Sc. with honors in Mathematics from Banaras Hindu University, India in 2005. He received his M.Tech. in Computer Technology and Applications from Delhi Technological University, India in 2010. Prior joining Indraprastha Institute of Information Technology Delhi (IIIT Delhi), India as an assistant professor, Dr Shah has received his Ph.D. in Computer Science from the National University of Singapore, Singapore. Currently, he is also working as a research fellow in living analytics research centre (LARC) at the Singapore Management University, Singapore. His research interests include the multimodal analysis of user-generated multimedia content in the support of social media applications, multimodal event detection and recommendation, and multimedia analysis, search, and retrieval. Dr Shah is the recipient of several awards, including the runner-up in the Grand Challenge competition of ACM International Conference on Multimedia. He is involved in reviewing of many top-tier international conferences and journals. He has published several research work in top-tier conferences and journals such as Springer MultiMedia Modeling, ACM International Conference on Multimedia, IEEE International Symposium on Multimedia, and Elsevier Knowledge-Based Systems.
This book presents a summary of the multimodal analysis of user-generated multimedia content (UGC). Several multimedia systems and their proposed frameworks are also discussed. First, improved tag recommendation and ranking systems for social media photos, leveraging both content and contextual information, are presented. Next, we discuss the challenges in determining semantics and sentics information from UGC to obtain multimedia summaries. Subsequently, we present a personalized music video generation system for outdoor user-generated videos. Finally, we discuss approaches for multimodal lecture video segmentation techniques. This book also explores the extension of these multimedia system with the use of heterogeneous continuous streams.|
Detailed summary of multimodal analysis of user-generated multimedia content literature
Proposed frameworks for several significant multimedia systems based on user-generated content
Leveraging multimodal information in solving several significant multimedia analytics problems
1 Introduction
1.1 Background and Motivation
1.2 Overview
1.3 Acronyms and Notations
1.4 Roadmap
2 Literature Review
2.1 Event Understanding
2.2 Tag Recommendation and Ranking
2.3 Soundtrack Recommendation for UGVs
2.4 Lecture Video Segmentation
3 Event Understanding
3.1 Introduction
3.2 System Overview
3.2.1 EventBuilder
3.2.2 EventSensor
3.3 Evaluation
3.3.1 EventBuilder
3.3.2 EventSensor
3.4 Summary
4 Tag Recommendation and Ranking
4.1 Introduction
4.1.1 Tag Recommendation
4.1.2 Tag Ranking
4.2 System Overview
4.2.1 Tag Recommendation
4.2.2 Random Walk based Relevance Scores
4.2.3 Fusion of Different Tag Recommendation Approaches
4.2.4 Tag Ranking
4.3 Evaluation
4.3.1 Tag Recommendation
4.3.2 Tag Ranking
4.4 Summary
5 Soundtrack Recommendation for UGVs
5.1 Introduction
5.1.1 Increasing Popularity of User-Generated Videos
5.1.2 Challenges with User-Generated Videos in Viewing and Sharing
5.1.3 Motivation for Generating Music Videos for Outdoor User-Generated Videos
5.2 Music Video Generation
5.2.1 Scene Moods Prediction Models
5.2.2 Music Retrieval Techniques
5.2.3 Automatic Music Video Generation Model
5.3 Evaluation
5.3.1 Dataset and Experimental Settings
5.3.2 Evaluation Metrics
5.3.3 Objective Evaluation
5.3.4 Subjective Evaluation
5.3.5 Experimental Results
5.3.6 Comparison with State-of-the-arts
5.3.7 Discussion of Results
5.4 Summary
6 Lecture Video Segmentation
6.1 Introduction
6.2 Lecture Video Segmentation
6.2.1 Prediction of Video Transition Cues using Supervised Learning
6.2.2 Computation of Text Transition Cues using N-gram based Language Model
6.2.3 Computation of SRT Segment Boundaries using the state-of-the-art
6.2.4 Computation of Wikipedia Segment Boundaries
6.2.5 Transition File Generation
<6.3 Evaluation
6.3.1 Dataset and Experimental Settings
6.3.2 Results from the ATLAS System
6.3.3 Results from the TRACE System
6.4 Summary
7 Conclusions and future work
This book presents a study of semantics and sentics understanding derived from user-generated multimodal content (UGC). It enables researchers to learn about the ways multimodal analysis of UGC can augment semantics and sentics understanding and it helps in addressing several multimedia analytics problems from social media such as event detection and summarization, tag recommendation and ranking, soundtrack recommendation, lecture video segmentation, and news video uploading.
Readers will discover how the derived knowledge structures from multimodal information are beneficial for efficient multimedia search, retrieval, and recommendation. However, real-world UGC is complex, and extracting the semantics and sentics from only multimedia content is very difficult because suitable concepts may be exhibited in different representations. Moreover, due to the increasing popularity of social media websites and advancements in technology, it is now possible to collect a significant amount of important contextual information (e.g., spatial, temporal, and preferential information). Thus, there is a need to analyze the information of UGC from multiple modalities to address these problems.
A discussion of multimodal analysis is presented followed by studies on how multimodal information is exploited to address problems that have a significant impact on different areas of society (e.g., entertainment, education, and journalism). Specifically, the methods presented exploit the multimedia content (e.g., visual content) and associated contextual information (e.g., geo-, temporal, and other sensory data). The reader is introduced to several knowledge bases and fusion techniques to address these problems.
This work includes future directions for several interesting multimedia analytics problems that have the potential to significantly impact society. The work is aimed at researchers in the multimedia field who would like to pursue research in the area of multimodal analysis of UGC.
Introduction.- Literature Review.- Event Understanding.- Tag Recommendation and Ranking.- Soundtrack Recommendation for UGVs.- Lecture Video Segmentation.- Adaptive News Video Uploading.- Conclusions and future work.
Rajiv Ratn Shah received his B.Sc. with honors in Mathematics from Banaras Hindu University, India in 2005. He received his M.Tech. in Computer Technology and Applications from Delhi Technological University, India in 2010. Prior joining Indraprastha Institute of Information Technology Delhi (IIIT Delhi), India as an assistant professor, Dr Shah has received his Ph.D. in Computer Science from the National University of Singapore, Singapore. Currently, he is also working as a research fellow in living analytics research centre (LARC) at the Singapore Management University, Singapore. His research interests include the multimodal analysis of user-generated multimedia content in the support of social media applications, multimodal event detection and recommendation, and multimedia analysis, search, and retrieval. Dr Shah is the recipient of several awards, including the runner-up in the Grand Challenge competition of ACM International Conference on Multimedia. He is involved in reviewingof many top-tier international conferences and journals. He has published several research work in top-tier conferences and journals such as Springer MultiMedia Modeling, ACM International Conference on Multimedia, IEEE International Symposium on Multimedia, and Elsevier Knowledge-Based Systems.
Über den Autor
Rajiv Ratn Shah received his B.Sc. with honors in Mathematics from Banaras Hindu University, India in 2005. He received his M.Tech. in Computer Technology and Applications from Delhi Technological University, India in 2010. Prior joining Indraprastha Institute of Information Technology Delhi (IIIT Delhi), India as an assistant professor, Dr Shah has received his Ph.D. in Computer Science from the National University of Singapore, Singapore. Currently, he is also working as a research fellow in living analytics research centre (LARC) at the Singapore Management University, Singapore. His research interests include the multimodal analysis of user-generated multimedia content in the support of social media applications, multimodal event detection and recommendation, and multimedia analysis, search, and retrieval. Dr Shah is the recipient of several awards, including the runner-up in the Grand Challenge competition of ACM International Conference on Multimedia. He is involved in reviewingof many top-tier international conferences and journals. He has published several research work in top-tier conferences and journals such as Springer MultiMedia Modeling, ACM International Conference on Multimedia, IEEE International Symposium on Multimedia, and Elsevier Knowledge-Based Systems.
Inhaltsverzeichnis
Introduction.- Literature Review.- Event Understanding.- Tag Recommendation and Ranking.- Soundtrack Recommendation for UGVs.- Lecture Video Segmentation.- Adaptive News Video Uploading.- Conclusions and future work.
Klappentext
This book presents a summary of the multimodal analysis of user-generated multimedia content (UGC). Several multimedia systems and their proposed frameworks are also discussed. First, improved tag recommendation and ranking systems for social media photos, leveraging both content and contextual information, are presented. Next, we discuss the challenges in determining semantics and sentics information from UGC to obtain multimedia summaries. Subsequently, we present a personalized music video generation system for outdoor user-generated videos. Finally, we discuss approaches for multimodal lecture video segmentation techniques. This book also explores the extension of these multimedia system with the use of heterogeneous continuous streams.
Detailed summary of multimodal analysis of user-generated multimedia content literature
Proposed frameworks for several significant multimedia systems based on user-generated content
Leveraging multimodal information in solving several significant multimedia analytics problems
