Skip to main content
SpringerLink
Log in

Learning JPEG Compression Artifacts for Image Manipulation Detection and Localization

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Detecting and localizing image manipulation are necessary to counter malicious use of image editing techniques. Accordingly, it is essential to distinguish between authentic and tampered regions by analyzing intrinsic statistics in an image. We focus on JPEG compression artifacts left during image acquisition and editing. We propose a convolutional neural network that uses discrete cosine transform (DCT) coefficients, where compression artifacts remain, to localize image manipulation. Standard CNNs cannot learn the distribution of DCT coefficients because the convolution throws away the spatial coordinates, which are essential for DCT coefficients. We illustrate how to design and train a neural network that can learn the distribution of DCT coefficients. Furthermore, we introduce Compression Artifact Tracing Network that jointly uses image acquisition artifacts and compression artifacts. It significantly outperforms traditional and deep neural network-based methods in detecting and localizing tampered regions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  • Agarwal, S., & Farid, H. (2018). A jpeg corner artifact from directed rounding of dct coefficients. Technical Report, TR2018-838, Dartmouth College

  • Amerini, I., Ballan, L., Caldelli, R., Del Bimbo, A., & Serra, G. (2011). A sift-based forensic method for copy-move attack detection and transformation recovery. IEEE Transactions on Information Forensics and Security, 6(3), 1099–1110.

    Article  Google Scholar 

  • Bammey, Q., Gioi, R. G. V., & Morel, J. M. (2020). An adaptive neural network for unsupervised mosaic consistency analysis in image forensics. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14194–14204.

  • Barni, M., Bondi, L., Bonettini, N., Bestagini, P., Costanzo, A., Maggini, M., et al. (2017). Aligned and non-aligned double jpeg detection using convolutional neural networks. Journal of Visual Communication and Image Representation, 49, 153–163.

    Article  Google Scholar 

  • Bas, P., Filler, T., & Pevnỳ, T. (2011). “break our steganographic system”: The ins and outs of organizing boss. In: International Workshop on Information Hiding, Springer, pp. 59–70.

  • Bayar, B., & Stamm, M. C. (2018). Constrained convolutional neural networks: A new approach towards general purpose image manipulation detection. IEEE Transactions on Information Forensics and Security, 13(11), 2691–2706.

    Article  Google Scholar 

  • Bi, X., Wei, Y., Xiao, B., & Li, W. (2019). Rru-net: The ringed residual u-net for image splicing forgery detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 0–0.

  • Bianchi, T., & Piva, A. (2012). Image forgery localization via block-grained analysis of jpeg artifacts. IEEE Transactions on Information Forensics and Security, 7(3), 1003–1017.

    Article  Google Scholar 

  • Boroumand, M., Chen, M., & Fridrich, J. (2018). Deep residual network for steganalysis of digital images. IEEE Transactions on Information Forensics and Security, 14(5), 1181–1193.

    Article  Google Scholar 

  • Butora, J., & Fridrich, J. (2020). Steganography and its detection in jpeg images obtained with the trunc quantizer. ICASSP 2020–2020 IEEE International Conference on Acoustics, pp. 2762–2766. IEEE: Speech and Signal Processing (ICASSP).

  • Chierchia, G., Poggi, G., Sansone, C., & Verdoliva, L. (2014). A bayesian-mrf approach for prnu-based image forgery detection. IEEE Transactions on Information Forensics and Security, 9(4), 554–567.

    Article  Google Scholar 

  • Choi, K. S., Lam, E. Y., & Wong, K. K. (2006). Source camera identification using footprints from lens aberration. Proceedings of SPIE, 6069, 172–179.

    Google Scholar 

  • Choi, C. H., Lee, H. Y., & Lee, H. K. (2013). Estimation of color modification in digital images by cfa pattern change. Forensic Science International, 226(1–3), 94–105.

    Article  Google Scholar 

  • Cozzolino, D., Poggi, G., & Verdoliva, L. (2015). Efficient dense-field copy-move forgery detection. IEEE Transactions on Information Forensics and Security, 10(11), 2284–2297.

    Article  Google Scholar 

  • Cozzolino, D., & Verdoliva, L. (2019). Noiseprint: A cnn-based camera model fingerprint. IEEE Transactions on Information Forensics and Security, 15, 144–159.

    Article  Google Scholar 

  • Dang-Nguyen, D.T., Pasquini, C., Conotter, V., & Boato, G. (2015). Raise: A raw images dataset for digital image forensics. In: Proceedings of the 6th ACM Multimedia Systems Conference, pp. 219–224.

  • De Carvalho, T. J., Riess, C., Angelopoulou, E., Pedrini, H., & de Rezende, Rocha A. (2013). Exposing digital image forgeries by illumination color classification. IEEE Transactions on Information Forensics and Security, 8(7), 1182–1194.

    Article  Google Scholar 

  • Dong, J., Wang, W., Tan, T. (2013). Casia image tampering detection evaluation database. In: 2013 IEEE China Summit and International Conference on Signal and Information Processing, IEEE, pp. 422–426.

  • Ferrara, P., Bianchi, T., De Rosa, A., & Piva, A. (2012). Image forgery localization via fine-grained analysis of cfa artifacts. IEEE Transactions on Information Forensics and Security, 7(5), 1566–1577.

    Article  Google Scholar 

  • Fridrich, J., & Kodovsky, J. (2012). Rich models for steganalysis of digital images. IEEE Transactions on Information Forensics and Security, 7(3), 868–882.

    Article  Google Scholar 

  • Fu, D., Shi, Y. Q., Su, W. (2007). A generalized benford’s law for jpeg coefficients and its applications in image forensics. In: Security, Steganography, and Watermarking of Multimedia Contents IX, International Society for Optics and Photonics, Vol. 6505, p. 65051L

  • Gloe, T., & Böhme, R. (2010). The ‘dresden image database’ for benchmarking digital image forensics. In: Proceedings of the 2010 ACM Symposium on Applied Computing, pp. 1584–1590.

  • Guan, H., Kozak, M., Robertson, E., Lee, Y., Yates, A. N., Delgado, A., Zhou, D., Kheyrkhah, T., Smith, J., & Fiscus, J. (2019). Mfc datasets: Large-scale benchmark datasets for media forensic challenge evaluation. In: 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW), IEEE, pp. 63–72.

  • He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778.

  • Hu, X., Zhang, Z., Jiang, Z., Chaudhuri, S., Yang, Z., & Nevatia, R. (2020). Span: Spatial pyramid attention network for image manipulation localization. In: European Conference on Computer Vision, Springer, pp. 312–328.

  • Huh, M., Liu, A., Owens, A., & Efros, A. A. (2018). Fighting fake news: Image splice detection via learned self-consistency. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 101–117.

  • Iakovidou, C., Zampoglou, M., Papadopoulos, S., & Kompatsiaris, Y. (2018). Content-aware detection of jpeg grid inconsistencies for intuitive image forensics. Journal of Visual Communication and Image Representation, 54, 155–170.

    Article  Google Scholar 

  • Kniaz, V. V., Knyaz, V., & Remondino, F. (2019). The point where reality meets fantasy: Mixed adversarial generators for image splice detection. In: Advances in Neural Information Processing Systems, pp. 215–226.

  • Korus, P. (2017). Digital image integrity-a survey of protection and verification techniques. Digital Signal Processing, 71, 1–26.

    Article  MathSciNet  Google Scholar 

  • Korus, P., & Huang, J. (2016). Multi-scale analysis strategies in prnu-based tampering localization. IEEE Transactions on Information Forensics and Security, 12(4), 809–824.

    Article  Google Scholar 

  • Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp. 1097–1105.

  • Kwon, M. J., Yu, I. J., Nam, S. H., & Lee, H. K. (2021). Cat-net: Compression artifact tracing network for detection and localization of image splicing. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 375–384.

  • Lam, E. Y., & Goodman, J. W. (2000). A mathematical analysis of the dct coefficient distributions for images. IEEE Transactions on Image Processing, 9(10), 1661–1666.

    Article  Google Scholar 

  • Lin, T. Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014). Microsoft coco: Common objects in context. In: European Conference on Computer Vision, Springer, pp. 740–755.

  • Lin, Z., He, J., Tang, X., & Tang, C. K. (2009). Fast, automatic and fine-grained tampered jpeg image detection via dct coefficient analysis. Pattern Recognition, 42(11), 2492–2501.

    Article  Google Scholar 

  • Liu, B., & Pun, C. M. (2020). Exposing splicing forgery in realistic scenes using deep fusion network. Information Sciences, 526, 133–150.

    Article  MathSciNet  Google Scholar 

  • Lukáš, J., & Fridrich, J. (2003) Estimation of primary quantization matrix in double compressed jpeg images. In: Proceedings of Digital Forensic Research Workshop, pp. 5–8.

  • Lukas, J., Fridrich, J., & Goljan, M. (2006). Digital camera identification from sensor pattern noise. IEEE Transactions on Information Forensics and Security, 1(2), 205–214.

    Article  Google Scholar 

  • Lyu, S., Pan, X., & Zhang, X. (2014). Exposing region splicing forgeries with blind local noise estimation. International Journal of Computer Vision, 110(2), 202–221.

    Article  Google Scholar 

  • Mahdian, B., & Saic, S. (2009). Using noise inconsistencies for blind image forensics. Image and Vision Computing, 27(10), 1497–1503.

    Article  Google Scholar 

  • Marra, F., Gragnaniello, D., Verdoliva, L., & Poggi, G. (2020). A full-image full-resolution end-to-end-trainable cnn framework for image forgery detection. IEEE Access, 8, 133488–133502.

    Article  Google Scholar 

  • Nam, S. H., Ahn, W., Yu, I. J., Kwon, M. J., Son, M., & Lee, H. K. (2020). Deep convolutional neural network for identifying seam-carving forgery. IEEE Transactions on Circuits and Systems for Video Technology

  • Ng, T. T., Chang, S. F., Sun, Q. (2004). A data set of authentic and spliced image blocks. Columbia University, ADVENT Technical Report 203-2004-3

  • Nikoukhah, T., Anger, J., Ehret, T., Colom, M., Morel, J. M., & Grompone von Gioi, R. (2019. Jpeg grid detection based on the number of dct zeros and its application to automatic and localized forgery detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 110–118.

  • Novozamsky, A., Mahdian, B., & Saic, S. (2020). Imd2020: A large-scale annotated dataset tailored for detecting manipulated images. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision Workshops, pp. 71–80.

  • Park, J., Cho, D., Ahn, W., & Lee, H. K. (2018). Double jpeg detection in mixed jpeg quality factors using deep convolutional neural network. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 636–652.

  • Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., & Antiga, L., et al. (2019). Pytorch: An imperative style, high-performance deep learning library. In: Advances in neural information processing systems, pp. 8026–8037.

  • Pham, N. T., Lee, J. W., Kwon, G. R., & Park, C. S. (2019). Hybrid image-retrieval method for image-splicing validation. Symmetry, 11(1), 83.

    Article  Google Scholar 

  • Piva, A. (2013). An overview on image forensics. International Scholarly Research Notices

  • Popescu, A. C., & Farid, H. (2004). Statistical tools for digital forensics. In: International Workshop on Information Hiding, Springer, pp. 128–147.

  • Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp. 91–99.

  • Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, pp. 234–241.

  • Swaminathan, A., Wu, M., & Liu, K. R. (2008). Digital image forensics via intrinsic fingerprints. IEEE Transactions on Information Forensics and Security, 3(1), 101–117.

    Article  Google Scholar 

  • Tralic, D., Zupancic, I., Grgic, S., & Grgic, M. (2013). Comofod-new database for copy-move forgery detection. In: Proceedings ELMAR-2013, IEEE, pp. 49–54.

  • Verdoliva, L. (2020). Media forensics and deepfakes: An overview. IEEE Journal of Selected Topics in Signal Processing, 14(5), 910–932.

    Article  Google Scholar 

  • Verma, V., Singh, D., & Khanna, N. (2020). Block-level double jpeg compression detection for image forgery localization. arXiv preprint arXiv:2003.09393

  • Wang, J., Sun, K., Cheng, T., Jiang, B., Deng, C., Zhao, Y., Liu, D., Mu, Y., Tan, M., & Wang. X., et al. (2020). Deep high-resolution representation learning for visual recognition. IEEE transactions on pattern analysis and machine intelligence

  • Wang, Q., Zhang, R. (2016). Double jpeg compression forensics based on a convolutional neural network. EURASIP Journal on Information Security,1, 23.

  • Wen, B., Zhu, Y., Subramanian, R., Ng, T. T., Shen, X., & Winkler, S. (2016). Coverage-a novel database for copy-move forgery detection. In: 2016 IEEE International Conference on Image Processing (ICIP), IEEE, pp. 161–165.

  • Wu, Y., Abd-Almageed, W., & Natarajan, P. (2018). Image copy-move forgery detection via an end-to-end deep neural network. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE, pp. 1907–1915.

  • Wu, Y., AbdAlmageed, W., & Natarajan, P. (2019). Mantra-net: Manipulation tracing network for detection and localization of image forgeries with anomalous features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9543–9552.

  • Ye, S., Sun, Q., & Chang, E. C. (2007). Detecting digital image forgeries by measuring inconsistencies of blocking artifact. In: 2007 IEEE International Conference on Multimedia and Expo, IEEE, pp. 12–15.

  • Yerushalmy, I., & Hel-Or, H. (2011). Digital image forgery detection based on lens and sensor aberration. International Journal of Computer Vision, 92(1), 71–91.

    Article  Google Scholar 

  • Yousfi, Y., & Fridrich, J. (2020). An intriguing struggle of cnns in jpeg steganalysis and the onehot solution. IEEE Signal Processing Letters.

  • Yu, I. J., Nam, S. H., Ahn, W., Kwon, M. J., & Lee, H. K. (2020). Manipulation classification for jpeg images using multi-domain features. IEEE Access, 8, 210837–210854.

    Article  Google Scholar 

  • Zampoglou, M., Papadopoulos, S., & Kompatsiaris, Y. (2017). Large-scale evaluation of splicing localization algorithms for web images. Multimedia Tools and Applications, 76(4), 4801–4834.

    Article  Google Scholar 

  • Zhou, P., Han, X., Morariu, V. I., & Davis, L. S. (2018) Learning rich features for image manipulation detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1053–1061.

Download references

Acknowledgements

This research was partially supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2021R1I1A1A01043600).

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea

    Myung-Joon Kwon & Changick Kim

  2. NAVER WEBTOON AI, Seongnam, South Korea

    Seung-Hun Nam

  3. Visual Display Business, Samsung Electronics Co., Ltd., Suwon, South Korea

    In-Jae Yu

  4. School of Computing, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea

    Heung-Kyu Lee

Authors
  1. Myung-Joon Kwon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seung-Hun Nam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. In-Jae Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Heung-Kyu Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Changick Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Changick Kim.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Communicated by Michael S. Brown.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kwon, MJ., Nam, SH., Yu, IJ. et al. Learning JPEG Compression Artifacts for Image Manipulation Detection and Localization. Int J Comput Vis 130, 1875–1895 (2022). https://doi.org/10.1007/s11263-022-01617-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-022-01617-5

Keywords

Search

Navigation