Hayley Hung

Hello, welcome to my webpage.

I am an Associate Professor at the Technical University of Delft, working on Socially Aware Surveillance Systems. I lead the recently formed Perceptive Computing Lab, which is part of the Pattern Recognition and Bioinformatics Group. Previously, I had a Marie Curie Postdoctoral Research fellowship on the AnaSID (Analysing Social Interactions from a Distance) project, in the Intelligent Autonomous Systems Group at the University of Amsterdam, working with Ben Krose. Before that, I was a PostDoc at IDIAP working on multimodal meeting analysis for modelling dominance and influence in meetings, working with Daniel Gatica-Perez. Before this, I was at Queen Mary University Vision Group as a PhD student supervised by Sean Gong.

Want a quick summary of what I do? Here are some links to related talks and videos:

Alternatively here are some press articles:

News

Researchs Interests

My main research interests are concerned with automatic human behaviour understanding, social signal processing, computer vision, multimodal processing and machine learning. Research Page...

PhDs, Postdocs, MSc students, Guests :

People I have supervised:

Publications

Journals, Editorials, and Books

E. Gedik and H. Hung, "Detecting Conversing Groups Using Social Dynamics fromWearable Acceleration: Group Size Awareness", to appear in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2019. pdf

L. Cabrera-Quiros and H. Hung, "A hierarchical approach for associating body-worn sensors to video regions in crowded mingling scenarios", in IEEE Transactions on Multimedia, 2018.

H. Hung, E. Gedik, and L. Cabrera-Quiros, "Complex Conversational Scene Analysis Using Wearable Sensing", in Multi-modal Behavior Analysis in the Wild: Advances and Challenges, Elseverie, Xavier Alameda-Pineda, Elisa Ricci, Nicu Sebe (Eds), 2019.

Y. Zhang, J. Olenick, C.H. Chang, S.W.J. Kozlowski, and H. Hung, "TeamSense: Assessing Personal Affect and Group Cohesion in Small Teams through Dyadic Interaction and Behavior Analysis with Wearable Sensors", in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2018.

L. Cabrera-Quiros*, A. Demetriou* , E. Gedik, L. v. d. Meij and H. Hung. "The MatchNMingle dataset: a novel multi-sensor resource for the analysis of social interactions and group dynamics in-the-wild during free-standing conversations and speed dates", in IEEE Transactions on Affective Computing, 2018. (*Authors contributed equally.) pdf . Data and Code

E. Gedik, L. Cabrera-Quiros, C. Martella, G. Englebienne, and H. Hung, "Towards Analyzing and Predicting the Experience of Live Performances with Wearable Sensing", in IEEE Transactions on Affective Computing, 2018

L. Zhang and H. Hung, "On Social Involvement in Mingling Scenarios: Detecting Associates of F-formations in Still Images", IEEE Transactions on Affective Computing 2018.

E. Gedik and H. Hung, "Personalised models for speech detection from body movements using transductive parameter transfer", Personal and Ubiquitous Computing, 2017 pdf

H. and H. Hung, "Is automatic facial expression recognition of emotions coming to a dead end? The rise of the new kids on the block", Image and Vision Computing, 2016 pdf

Sebastiano Vascon, Eyasu Z. Mequanint, Marco Cristani, Hayley Hung, Marcello Pelillo, Vittorio Murino, "Detecting conversational groups in images and sequences: A robust game-theoretic approach", Computer Vision and Image Understanding, Volume 143, February 2016, Pages 11-24.

H. Hung and G. Toderici,"Editoral Note to Special Section on Extended Best Paper Candidates of ACM Multimedia 2014", ACM Transactions on Multimedia, 2015.

G. Li, S. Whiteson, B. Knox, and H. Hung, "Using Informative Behavior to Increase Engagement While Learning from Human Reward", Journal on Autonomous Agents and Multi-Agent Systems (JAAMAS), 2015 pdf

H. Gunes* and H. Hung*, "Emotional and Social Signals: A Neglected Frontier in Multimedia Computing?", IEEE Multimedia Magazine, Visions and Views section, April/June 2015. pdf *both authors contributed equally to this article.

Salah, A.A., H. Hung, O. Aran, H. Gunes, "Creative applications of human behavior understanding", HBU 2013, LNCS 8212, pp.1-14, Springer Verlag, Berlin Heidelberg.

H. Hung, "Deception in Multiparty Contexts", IEEE Intelligent Systems, Trends & Controversies, "Computational Deception and Noncooperation", Edited by Anton Nijholt, 2012:pdf

H. Hung, Y. Huang, G. Friedland, D. Gatica-Perez, "Estimating Dominance In Multi-Party Meetings Using Speaker Diarization", IEEE Transactions on Audio, Speech and Language Processing, Volume 19, Issue 4, pages 847-860, May 2011, :pdf.

G. Friedland, C. Yeo, and H. Hung, "Dialocalization: Acoustic Speaker Diarization and Visual Localization as Joint Optimization Problem", ACM Transactions on Multimedia Computing, Communications and Applications, Special Issue on Sensor Fusion, Vol. 6, No. 4, November 2010.

H. Hung, G. Friedland, and C. Yeo, "Computationally Efficient Clustering of Audio-Visual Meeting Data", in Multimedia Interaction and Intelligent User Interfaces: Principles, Methods and Applications in the Series Advances in Pattern Recognition, 25-59, Springer, Shao, L.; Shan, C.; Luo, J.; Etoh, M. (Eds.), November 15, 2010.pdf

H. Hung and D. Gatica-Perez, "Estimating Cohesion in Small Groups using Audio-Visual Nonverbal Behavior", IEEE Transactions on Multimedia, Vol. 12, No. 6, Oct. 2010.pdf.

D. Jayagopi, H. Hung, C. Yeo and D Gatica-Perez, "Modeling Dominance in Group Conversations using Non-verbal Activity Cues",IEEE Trans. on Audio, Speech, and Language Processing, Special Issue on Multimodal Processing for Speech-based Interactions, Vol. 17, No. 3, pp. 501-513. Mar. 2009 pdf

Refereed Conferences and Workshops

L Cabrera-Quiros, E Gedik, and H Hung, "Transductive Parameter Transfer, Bags of Dense Trajectories and MILES for No-Audio Multimodal Speech Detection", in Proceedings of the MediaEval Workshop, 2018

G. Murray, H. Hung, J. Keyton, C. Lai, N. Lehmann-Willenbrock, and C. Oertel, "Group Interaction Frontiers in Technology", Proceedings of the International Conference on Multimodal Interaction, 2018.

L Cabrera-Quiros, E Gedik, and H Hung, "No-Audio Multimodal Speech Detection in Crowded Social Settings task at MediaEval 2018", in Proceedings of the MediaEval Workshop, 2018

B. Dudzik, J. Broekens, M. Neerincx, J. Olenick, C.H. Chang, S.W.J. Kozlowski, and H. Hung, "Discovering digital representations for remembered episodes from lifelog data", Proceedings of the International Conference on Multimodal Interaction workshop on Modeling Cognitive Processes from Multimodal Data, 2018

B. Dudzik, H. Hung, M. Neerincx, and J. Broekens, "Artificial Empathic Memory: Enabling Media Technologies to Better Understand Subjective User Experience", Proceedings of the ACM Multimedia Workshop on Understanding Subjective Attributes of Data, with the Focus on Evoked Emotions, 2018.

S. Tan, D. Tax, and H. Hung, "Improving Temporal Interpolation of Head and Body Pose using Gaussian Process Regression in a Matrix Completion Setting", Proceedings of the International Conference on Multimodal Interaction workshop on Group Interaction Frontiers in Technology, 2018.

Y. Zhang and H. Hung, "Using topic models to mine everyday object usage routines through connected IoT sensors", Proceedings of the 8th International Conference on the Internet of Things, 2018

Y. Zhang, J. Olenik, Chu-Hsiang Chang, S. W. J. Kozlowski, and H. Hung, "The I in Team: Mining Personal Social Interaction Routine with Topic Models from Long-Term Team Data", ACM International Conference on Intelligent User Interfaces (IUI), 2018.

M. Nanninga, Y. Zhang, N. Lehmann-Willenbrock, Z. Szlavik, and H. Hung, "Estimating verbal expressions of task and social cohesion in meetings by quantifying paralinguistic mimicry", ICMI 2017

L. Cabrera-Quiros, E. Gedik, and H. Hung, "Estimating Self-Assessed Personality from Body Movements and Proximity in Crowded Mingling Scenarios", in International Conference on Multimodal Interaction (ICMI), 2016. (oral presentation)

L. Cabrera-Quiros and H. Hung, "Who is where? Matching People in Video to Wearable Acceleration During Crowded Mingling Events", ACM MM 2016.

E. Gedik and H. Hung, "Speech Detection from Body Movements in Crowded Environments: A Transfer Learning Approach", in the ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) Adjunct Proceedings, 2016

L. Zhang and H. Hung, "Beyond F-formations: Determining Social Involvement in Free Standing Conversing Groups from Static Images", CVPR 2016.

C. Martella*, E. Gedik*, L. Cabrera-Quiros*, G. Englebienne, and H. Hung, "How Was It? Exploiting Smartphone Sensing to Measure Implicit Audience Responses to Live Performances", ACM MM 2015 (oral) *These authors contributed equally to this article.pdf *These authors contributed equally to this article.

Rudolph Triebel, Kai Arras, Rachid Alami, Lucas Beyer, Stefan Breuers, Raja Chatila, Mohamed Chetouani, Daniel Cremers, Vanessa Evers, Michelangelo Fiore, Hayley Hung, Omar A. Islas Ramírez, Michiel Joosse, Harmish Khambhaita, Tomasz Kucner, Bastian Leibe, Achim J. Lilienthal, Timm Linder, Manja Lohse, Martin Magnusson, Billy Okal, Luigi Palmieri, Umer Rafi, Marieke van Rooij, Lu Zhang, SPENCER: A Socially Aware Service Robot for Passenger Guidance and Help in Busy Airports, In Proc. Field and Service Robotics (FSR), 2015

K. Schellekens, D. Day, E. Gedik, L. Cabrera-Quiros, C. Martella, H. Hung, and E. Giaccardi, "Impact of connected objects on social encounters", Participatory Innovation Conference, The Hague, 2015 pdf

G. Li, H. Hung, S. Whiteson, "A Large-Scale Study of Agents Learning from Human Reward", in International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2015)

Sebastiano Vascon, Eyasu Zemene Mequanint, Marco Cristani, Hayley Hung, Marcello Pelillo and Vittorio Murino, "A Game-Theoretic Probabilistic Approach for Detecting Conversational Groups ", ACCV, 2014

H. Hung, G. Englebienne, and L. Cabrera-Quiros, "Detecting Conversing Groups with a Single Worn Accelerometer ", in the ACM International Conference on Multimodal Interaction, 2014 pdf

G. Li, H. Hung, S. Whiteson, and W. B. Knox, "Learning from Human Reward Benefits from Socio-competitive Feedback ", in the International Conference on Development and Learning and on Epigenetic Robtics (ICDL-Epirob) , 2014

G. Li, H. Hung, S. Whiteson, and W. B. Knox, "Leveraging social networks to motivate humans to train agents ", in International Conference on Autonomous Agents and Multiagent Systems, Minnesota,May, 2014

H.Hung and G. Englebienne, "Systematic Evaluation of Social Behaviour Modelling with a Single Accelerometer", in the ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp), 2013

H. Hung, G. Englebienne, and J. Kools, "Classifying Social Action with a Single Accelerometer", in the ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp), 2013 (oral presentation) pdf

F. Setti*, H. Hung*, and M. Cristani*, "Group Detection in Still Images by F-formation Modeling: A Comparative Study", Invited Paper, in the International Workshop on Image and Audio Analysis for Multimedia Interactive Services (WIA2MIS), 2013 (oral presentation).pdf *all authors contributed equally.

G. Li, H. Hung, S.Whiteson, W.B. Knox, "Using Informative Behavior to Increase Engagement in the TAMER Framework", International Conference on Autonomous Agents and Multiagent Systems, Minnesota,May, 2013 (oral presentation). pdf

G. Englebienne and H. Hung, "Mining for Motivation: Using a single wearble accelerometer to detect people's interests ", in ACM Multimedia Workshop on Interactive Multimedia on Mobile and Portable Devices, Nara, Japan, 2012.pdf

H. Hung, J.-M. Odobez, D. Gavrila, "International Workshop on Interactive Human Behavior Analysis in Open or Public Spaces", workshop proceedings of the International Joint Conference on Ambient Intelligence, Amsterdam, 2011.

H. Hung, J.-M. Odobez, D. Gavrila, "Interactive Human Behavior Analysis in Open or Public Spaces", in the International Joint Conference on Ambient Intelligence, Amsterdam, 2011.

A. Veenstra and H. Hung, "Do They Like Me? Using Video Cues to Predict Desires during Speed-dates", IEEE International Conference on Computer Vision Workshops,p. 838-845, Barcelona, Spain, 2011. (Oral Presentation): pdf

N. Raiman, G. Englebienne, and H. Hung, "Move, and I will tell you who you are: Detecting deceptive roles in low-quality data", International Conference on Multimodal Interaction, Alicante, Spain, November 2011: pdf corpus

H. Hung and B. Krose, "Detecting F-formations as Dominant Sets", in International Conference on Multimodal Interaction, Alicante, Spain, November 2011.(Oral Presentation): pdf
Nominated for Outstanding Paper. Data available here

H. Hung and C. Jacquemin, "Encounter (Resonances)", in ACM Multimedia, Florence, Italy,1421-1424, October, 2010:pdf video.

H. Hung and G. Chittaranjan, "The Idiap Wolf Corpus: Exploring group behaviour in a competitive role-playing game", in ACM Multimedia, Florence, Italy, 879-882, October, 2010:pdf corpus.

Oya Aran, Hayley Hung, Daniel Gatica-Perez, "A Multimodal Corpus for Studying Dominance in Small Group Conversations", LREC workshop on Multimodal Corpora: Advances in Capturing, Coding and Analyzing Multimodality, Malta, May 2010:pdf

G. Chittaranjan and H. Hung, "Are You A Werewolf? Detecting Deceptive Roles And Outcomes in a Conversational Role-Playing Game", in ICASSP, Dallas, Texas, USA, March 14-19, 2010:pdf

G. Friedland, C. Yeo and H. Hung, "Visual Speaker Localization Aided by Acoustic Models", ACM Multimedia, Beijing, China, October 19-24, 2009.

S.O. Ba, H. Hung and J.-M. Odobez. "Visual activity context for focus of attention estimation in dynamic meetings", International Conference on Multimedia and Expo (ICME), June 2009:pdf

G. Friedland, H.Hung, C.Yeo,"Multi-Modal Speaker Diarization of Real-World Meetings using Compressed-Domain Video Features", International Conference on Acoustics, Speech and Signal Processing, Taipei, April 2009:pdf .

D. Jayagopi, H.Hung, C. Yeo, and D. Gatica-Perez, "Predicting the dominant clique in meetings through fusion of nonverbal cues", in Proc. ACM Multimedia, Vancouver, Canada, October 27-November 1, 2008:pdf.

H.Hung, D. Jayagopi, S. Ba, J-M. Odobez, and D. Gatica-Perez, "Investigating automatic dominance estimation in groups from visual attention and speaking activity", in the International Conference on Multimodal Interfaces (ICMI 2008), Chania, Greece, October 20-22, 2008:pdf (Oral Presentation)

H.Hung and G. Friedland, "Towards Audio-Visual On-line Diarization Of Participants In Group Meetings", in Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications in conjunction with ECCV, Marseille, France, October 18, 2008:pdf.

H.Hung and D. Gatica-Perez, "Identifying Dominant People in Meetings from Audio-Visual Sensors", in the International Conference on Automatic Face and Gesture Recognition, Amsterdam, The Netherlands, September 17-19, 2008:pdf (Oral Presentation)

H. Hung, Y. Huang, C. Yeo, D. Gatica-Perez, "Associating Audio-Visual Activity Cues in a Dominance Estimation Framework", First IEEE Workshop on CVPR for Human Communicative Behavior Analysis, Anchorage, Alaska, Monday June 28th, 2008:pdf

H. Hung, Y. Huang, G. Friedland, and D. Gatica-Perez, "Estimating The Dominant Person In Multi-Party Conversations Using Speaker Diarization Strategies", International Conference on Acoustics, Speech and Signal Processing, Las Vegas, March 2008:pdf

H. Hung, D. Jayagopi, C. Yeo, G. Friedland, S. Ba, J-M. Odobez, N. Mirghafori, K. Ramchandran, D. Gatica-Perez, "Using Audio and Video Features to Classify the Most Dominant Person in a Group Meeting", Proc. ACM Multimedia, Augsburg, Germany, September 2007 : pdf

H. Hung and S. Gong, "Identification of human interaction from a distance using salient body behaviour modelling", Proc. SPIE Conference on Optics and Photonics for Counter-Terrorism and Crime-Fighting, Stockholm, September 2006,Vol. 6402, 640203.(Oral Presentation)

H. Hung and S. Gong, "Detecting and Quantifying Unusual Interactions by Correlating Salient Motion ", Proc. IEEE International Conference on Advanced Video and Signal-Based Surveillance (AVSS'05), Como, Italy, p46-51, pdf (Oral Presentation)

H. Hung and S. Gong, "Quantifying Temporal Saliency", Proc. British Machine Vision Conference (BMVC'04), Kingston, UK, p727-736, pdf

PhD Thesis

Hayley Hung, "FROM VISUAL SALIENCY TO VIDEO BEHAVIOUR UNDERSTANDING", PhD Thesis, Department of Computer Science, Queen Mary University, 2007.

Technical Reports

H. Hung and S. O. Ba, SPEECH/NON-SPEECH DETECTION IN MEETINGS FROM AUTOMATICALLY EXTRACTED LOW RESOLUTION VISUAL FEATURES, Idiap Research Report, 2010 pdf

D. Jayagopi, H. Hung, C. Yeo, D. Gatica-Perez, "Modeling Dominance in Group Conversations using Non-verbal Activity Cues", IDIAP Research Report rr-07-78, 2007.

T. Xiang, M. Aurnhammer, S. Gong, H. Hung and A. Graves. "INSIGHT and Beyond: CCTV video data analysis and evaluation (Phase I)". Technical Report, Department of Computer Science, Queen Mary, University of London, July 2004.

H Hung, "Towards modelling the Semantics of Natural Human Body Movement", PhD-MPhil Transfer report, 2003

Awards and scholarships

2014: ICME Outstanding Reviewer Award

2009: First Prize, The IET Written Premium Competition, "Who's the Boss?: From Social Pyschology to Social Signal Processing",.

2006: Second Prize, The IET Written Premium Competition, "Spotting the bad guys - A journey from surveillance video to automated crime fighting" .

2005: RAeng travel bursary.

1999: Lucent Technologies Global Science Scholarship.

1998: Sponsorship from Qinetiq Ltd during first degree.

1998: West Midlands Runner Up Young Engineer for Britain Award and Year in Industry Scheme West Midlands Runner Up Bursary Award for design of an automatic self-aligning antenna.

Other Projects