Forensic Roadmap

My thesis research focuses on Multimedia Forensics. Given a digital image, my work aims to answer a number of forensic questions related to origin and authenticity of the image such as

  • How was the image captured?
  • Was it captured using a digital standalone camera, cell phone camera, digital scanner, or was it generated using computer graphics?
  • If the image was captured using a camera, which brand or model of camera was used to capture the image?
  • Has the image been tampered or manipulated after capture?
  • Does it contain any hidden information or stego data?

We classify these forensic questions into the following categories:

  • Component Forensics aims at identifying the algorithms and parameters of an imaging device based on its output data
  • Acquisition forensics aims to determine if the image was captured using a digital camera, cell phone camera, scanner, or was computer generated

In our work, we introduce a new set of forensic techniques based on Intrinsic Fingerprints to answer the forensic questions posed above. These techniques constitute what we refer to as Non-Intrusive Forensics as the forensic analysis is conducted solely based on the given image without any additional information. For our work on non-intrusive forensics, we focus on digital standalone cameras, scanners, and cell phone cameras and these devices collectively constitute the source of more than 99% of digital images in the modern era.

Next, we consider an alternate scenario for forensic analysis - semi non-intrusive forensics. Suppose the forensic analyst has access to the device at hand, he/she can design appropriate inputs to be fed into the device so as to collect more forensic evidence about the device components. We study the problem of semi non-intrusive forensics with a specific example from digital cameras. Further details can be found here.

We also develop a unified theoretical framework for forensic analysis to characterize the limits of multimedia forensics. More information about this framework is available here.


Non-Intrusive Component Forensics

Visual sensors have experienced a tremendous amount of growth and are becoming increasingly popular every year. Such rapid technology development and widespread use has led to a number of new problems related to protecting intellectual property rights, handling patent infringements, authenticating acquisition source, and identifying content manipulations. This work introduces non-intrusive component forensics as a new methodology for forensic analysis. Non-intrusive component forensics aims at identifying the algorithms and parameters employed inside the various processing modules of a digital device, using only the sample data collected from device outputs without breaking the device apart. In this work, we illustrate the proposed forensic methodology for visual sensors, and present novel techniques to estimate the parameters of the important camera components such as color filter array and color interpolation algorithms.

The results obtained from such analysis are useful in a number of scenarios

  • Infringement/Licensing forensics for studying the similarities between the technologies employed by different camera models
  • Camera Identification for determining the brand/make that was used to capture an image
  • Cut-Paste tampering detection by studying possible discrepancies in the estimated parameters from different regions of the image

Details can be found in

[TIFS07] Ashwin Swaminathan, Min Wu, and K. J. Ray Liu, Non-Intrusive Component Forensics of Visual Sensors Using Output Images, IEEE Transactions on Information Forensics and Security, vol. 2, no. 1, pp. 91-106, March 2007. [pdf] [url]

[ICASSP06] Ashwin Swaminathan, Min Wu, and K. J. Ray Liu, Non-Intrusive Forensic Analysis of Visual Sensors Using Output Images, IEEE Conference on Acoustic, Speech and Signal Processing (ICASSP), vol. 5, pp. 401-404, Toulouse, France, May 2006. [pdf] [presentation]

[CISS06] Ashwin Swaminathan, Min Wu, and K. J. Ray Liu, Component Forensics of Digital Cameras: A Non-Intrusive Approach, Proceedings of Conference on Information Sciences and Systems (CISS), invited paper, pp. 1194-1199, Princeton, NJ, March 2006. [pdf] [presentation]

[InvDisc06] Ashwin Swaminathan, Min Wu, and K. J. Ray Liu, Component Forensics of Visual Sensors and Devices, filed March 2006.

[InvitedTalk06] Ashwin Swaminathan, Component Forensics of Visual Sensors, at Hewlett-Packard Labs, Palo Alto, CA, August 2006. [presentation]


Forensic Analysis via Intrinsic Fingerprints


A camera leaves intrinsic fingerprint traces in all the images that it captures. Building upon component forensics, we develop techniques to estimate the intrinsic fingerprint traces left behind by the camera on the final output image, and use these traces to answer a number of questions related to origin and authenticity of a digital image. We consider the direct output images of a camera as authentic, and characterize its properties by modeling the imaging process. Any further processing applied to the camera output is modeled as a manipulation block and its coefficients are obtained using blind deconvolution. These estimated coefficients are then used to identify possible manipulations. We demonstrate the widespread effectiveness of the proposed technique for numerous applications:

  • Tampering forensics to identify image manipulations
  • Production forensics to trace the source of the digital image
  • Blind image steganalysis to identify hidden information in multimedia data

Details can be found in

[ICIP06] Ashwin Swaminathan, Min Wu, and K. J. Ray Liu, Image Tampering Identification using Blind Deconvolution, Proceedings of the IEEE International Conference on Image Processing (ICIP), pp. 2311-2314, Atlanta, GA, October 2006. [pdf] [presentation]

[SPIE07a] Ashwin Swaminathan, Min Wu, and K. J. Ray Liu, Intrinsic Fingerprints for Image Authentication and Steganalysis, IS&T SPIE Conference on Security, Steganography and Watermarking of Multimedia Contents IX, San Jose, CA, January 2007. [pdf] [presentation]

[TIFS08] Ashwin Swaminathan, Min Wu, and K. J. Ray Liu, Forensic Analysis via Intrinsic Fingerprints, IEEE Transactions on Information Forensics and Security, to appear March 2008. [pdf]


Scanner Forensics

While cameras provide digital reproduction of natural scenes, scanners are often used to capture hard-copy art in more controlled environment. In this work, we propose a new technique of utilizing intrinsic sensor noise features for non-intrusive scanner forensics for applications in

  • Verifying the source and integrity of digital scanned images
  • Constructing a robust scanner identifier to determine the model of the scanner used to capture a scanned image
  • Differentiating scanned images from camera taken photographs and computer generated graphics
  • Tampering forensic analysis on scanned images to detect post-processing operations on scanned images

The proposed methods are also robust against a number of post-processing operations on scanned images.

Further details can be found in

[SPIE07b] Hongmei Gou, Ashwin Swaminathan, and Min Wu, Robust Scanner Identification based on Noise Features, IS&T SPIE Conference on Security, Steganography and Watermarking of Multimedia Contents IX, San Jose, CA, January 2007. [pdf] [presentation]

[ICIP07b] Hongmei Gou, Ashwin Swaminathan, and Min Wu, Noise Features for Image Tampering Detection and Steganalysis, IEEE Conference on Image Processing (ICIP), San Antonio, TX, September 2007. [pdf] [presentation]

[TIFS07] Hongmei Gou, Ashwin Swaminathan, and Min Wu, Intrinsic Sensor Noise Features for Scanner and Scanned Image Forensics, to be submitted to IEEE Transactions on Information Forensics and Security, May 2008.

Cell Phone Camera Forensics

In our work with cell phone cameras, we extend several of the proposed techniques from standalone digital cameras and scanners to cell phone cameras. Methods used include estimating a device's color interpolation coefficients and noise feature parameters. Robustness to post-camera operations such as digital zoom and JPEG compression is also examined.

The work on cell phone cameras is a joint work with Christine McKay as part of the University of Maryland, MERIT program for undergraduate students. Under my mentoring, Christine was awarded the best student researcher of Summer 2008. Further details about the award can be found here.

More information about this work can be found in

[MERIT07] Christine McKay, Digital Image Detective: Forensic Analysis to Identify Image Source, MERIT Fair 2007. Mentored by Ashwin Swaminathan, Hongmei Gou, and Min Wu. Won the best overall first place in MERIT fair. [report] [poster] [presentation]

[ICASSP08b] Christine McKay, Ashwin Swaminathan, Hongmei Gou, and Min Wu, Image Acquisition Forensics: Forensic Analysis to Identify Imaging Source, accepted to the IEEE International Conference on Acoustic, Speech, and Signal Processing, Las Vegas, NV, March 2008. [pdf] [presentation]

For our current work, we have focussed on standalone digital cameras, scanners, and cell phone cameras. For future work, we plan to explore other types of imaging devices such as video cameras and webcameras and display devices such as monitors and projectors.



Semi Non-Intrusive Forensic Analysis of Digital Cameras

In this work, we consider an alternate scenario for forensic analysis - semi non-intrusive forensics. Suppose the forensic analyst has access to the device at hand, he/she can design appropriate inputs to be fed into the device so as to collect more forensic evidence about the device components. We study the problem of semi non-intrusive forensics with a specific example from digital cameras. Based on a detailed modeling of the imaging process, we design optimal inputs to obtain a better understanding of the color interpolation and white balancing algorithms employed in cameras. Our simulation results show that the overall confidence in estimating component parameters can be improved significantly by this approach compared to non-intrusive forensics.
Further details can be found in

[ICASSP07a] Ashwin Swaminathan, Min Wu, and K. J. Ray Liu, Optimization of Input Pattern for Semi Non-Intrusive Component Forensics of Digital Cameras, IEEE Conference on Acoustic, Speech and Signal Processing (ICASSP), Honolulu, HI, April 2007. [pdf] [presentation]


Theoretical Framework for Forensic Analysis

In this work, we propose a theoretical foundation to examine the performance limits of component forensics. Using ideas from estimation theory, we define formal notions of identifiability of components in the information processing chain, and present methods to quantify the accuracies at which the component parameters can be estimated. We show that the parameters of certain device components can be identified only in controlled settings through semi non-intrusive forensics, while the parameters of some others can be estimated directly from sample data via complete non-intrusive analysis. Building upon the proposed theoretical framework, we devise methods to improve the accuracies of component parameter estimation for a wide range of forensic applications.
More details can be found in:

[MMSP07] Ashwin Swaminthan, Min Wu, and K. J. Ray Liu,
A Component Estimation Framework for Information Forensics, IEEE Workshop on Multimedia Signal Processing, Crete, Greece, October 2007. [pdf] [presentation]

[ICASSP08a] Ashwin Swaminathan, Min Wu, and K. J. Ray Liu,
A Pattern Classification Framework for Theoretical Analysis of Component Forensics, accepted to the IEEE International Conference on Acoustic, Speech, and Signal Processing, Las Vegas, NV, March 2008. [pdf] [presentation]