iPrivacy: Privacy-Preserving Image Sharing via Multi-Party Privacy Setting and Negotiation

  • Research Problems: With the growing ubiquity of high-speed Internet, smartphones and other mobile devices, photo (images) sharing has now become very popular in social networks like Facebook, Instagram and Foursquare. By default, the shared photos can be seen by anyone in social networks. Since photos can intuitively tell when and where a special moment took place, who participated and what were their relationships, photos can reveal much of users' personal and social environments and their private lives. Thus, privacy protection is a critical issue to be addressed during photo sharing. However, many people especially young users of social networks often share private photos about themselves, their friends and classmates without being aware of the potential impact on their future lives caused by unwanted disclosure and privacy violations. Aiming to provide privacy protection, many photo sharing sites nowadays allow users to manually set up coarse-grained privacy settings (or preferences) to specify whether a photo is public, private or visible to their family members or friends. Recently, some photo sharing platforms also allow users to specify more detailed and advanced privacy settings. However, due to the lack of privacy knowledge, it would not be easy for common users to correctly configure privacy settings to achieve their desired levels of privacy protection; also, due to the large number of photos to be shared and the tedious steps needed for privacy settings, some users may not even be willing to spend extra time on such settings.
  • Research Goals: (a) the first goal of our project is to develop new framework to achieve fast detection and recognition of large-scale object classes from massive images being shared; (b) the second goal of our project is to develop new techniques that are able to automatically recommend personalized privacy settings for each single photo being shared; (c) the second goal of our project is to automatically harmonize privacy settings that involve multiple parties owning the same photo.
  • Proposed Researches: (a) Identifying Multi-Party Privacy-Sensitive Object Classes: Multi-party privacy-sensitive (MPPS) objects refer to photos that contain sensitive information and have multiple human subjects in them. Our first task aims to automatically identify a complete list of MPPS objects from shared photos and study their hidden dependencies regarding privacy settings through the photo owners' past behavior (i.e., previous privacy settings). (b) Fast Detection of Large-Scale MPPS Object Classes: The second task aims to address efficiency and scalability issues during the MPPS detection (at the time for photo sharing) so as to provide real-time services to end users adopting our system. Specifically, we plan to build a visual hierarchy to organize the potentially large number of MPPS object classes in a coarse-to-fine fashion, and then construct max-margin tree classifiers over the visual hierarchy for fast detection. (c) Automatic Recommendation of Personalized Privacy Settings: Based on the MPPS object classes obtained from the previous tasks, the third task aims to automatically generate personalized privacy settings for users. Specifically, we will design novel policy mining, privacy evolution modeling and multi-party policy harmonization algorithms.

    • Current Project Achievements

    System Flowchart for iPrivacy

    Some Experimental Results

  • Privacy-Sensitive Object Detection in Real-Time and Image Privacy Protection via Simple Blurring:
    (a) original images; (b) extracting masks of human objects via deep learning; (c) extracted human objects; (d) face detection for identifying privacy-sensitive human objects; (e) simple blurring of privacy-sensitive human objects; (f) shared images with privacy protection via simple blurring.
  • Image Privacy Protection via Object Replacement:
    (a) original images; (b) face detection; (c) human object extraction according to the detected faces and human models; (d) privacy protection via simple blurring; (e) privacy protection by replacing the privacy-sensitive human objects with virtual objects.

    • Preliminary Publications:

  • J. Yu, B. Zhang, Z. Kuang, D. Lin, J. Fan, ``iPrivacy: Image privacy protection by identifying sensitive objects via deep multi-task learning", IEEE Transactions on Information Forensics and Security, vol. 13, no.2, 2017.
  • Y. Nakashima, N. Babaguchi, J. Fan, ``Privacy protection for social video via background estimation and CRF-based videographers intention modeling", IEICE Trans. 99-D(4), pp.1121-1233, 2016.
  • B. Zhang, H. Luo, J. Fan, ``Statistical modeling for automatic image indexing and retrieval", Neurocomputing, vol. 207, pp. 105-119, 2016.
  • J. Fan, H. Luo, M. Hacid, E. Bertino, A novel approach for privacy-preserving video sharing, ACM CIKM, pp.609-616, 2005.
  • H. Luo, J. Fan, X. Lin, A. Zhou, E. Bertino, A distributed approach to enabling privacy-preserving model-based classifier training, Knowledge Information System, vol.20, no.2, pp.157-185, 2009.
  • J. Peng, N. Babaguchi, H. Luo, Y. Gao, J. Fan, Constructing distributed hippocratic video databases for privacy-preserving online patient training and counseling, IEEE Trans. Information Technology in Biomedicine, vol.14, no.4, pp.1014-1026, 2010.
  • Y. Nakashima, N. Babaguchi, J. Fan, Automatic generation of privacy-protected videos using background estimation, IEEE ICME, pp.1-6, 2011.
  • Y. Nakashima, N. Babaguchi, J. Fan, Intended human object detection for automatically protecting privacy in mobile video surveillance, Multimedia Systems, vol.18, no.2, pp.157-173, 2012.

    • If we know what we were doing, it wouldn't be research, would it? ---Albert Einstein(1879-1955)---