ISITA2010, Taichung, Taiwan, October 17-20, 2010

Digital Watermarking Method for Tamper Detection and Recovery of JPEG Images
Motoi Iwata, Tomoki Hori, Akira Shiozaki, and Akio Ogihara
Graduate School of Engineering, Osaka Prefecture University, Osaka, Japan Email: iwata@cs.osakafu-u.ac.jp and recovery of BMP images based on a Reed-Solomon code (RS code) and RSA cryptography. Park, et al. [7] proposed the self-embedding method for BMP images like [2]. This method embedded the permuted small-sized image etc. into the rst and the second LSB of an original image, and then didnt have the robustness against JPEG compression. Luo, et al. [8] and Chang, et al. [9] also proposed the self-embedding method using LSB replacement of BMP images. Then both of them were not robust against JPEG compression as well as Parks method. In this manuscript, we propose a new method for tamper detection and recovery of JPEG images by using a RS code, where this idea is based on Minamis method [6]. The proposed method embeds the check symbols corresponding to an information-reduced original image into the LSBs of the quantized DCT coefcients in a JPEG image directly. Therefore the method completely extracts the embedded data from watermarked JPEG images. The rest of this manuscript consists of four sections. We describe the summary of JPEG compression in Sec. II. Then we describe the detail of the proposed method in Sec. III, and show and discuss the performance of the proposed method in Sec. IV. Finally we conclude this manuscript in Sec. V. II. JPEG C OMPRESSION Figure 1 shows the ow of JPEG compression. JPEG compression process consists of conversion of an RGB color image to a YCb Cr color image, composition of minimum coding units (MCU), 2-dimensional discrete cosine transform (2-D DCT), quantization of DCT coefcients, runlength coding and Huffman coding [11], [12]. A. Conversion of an RGB color image to a YCb Cr color image An RGB color image is converted to a YCb Cr color image using the following equations: Y = 0.2990R + 0.5870G + 0.1140B 128, Cb = 0.1687R 0.3313G + 0.5000B, Cr = 0.5000R 0.4187G 0.0813B, (1) (2) (3)

AbstractDigital watermarking is a technique that embeds additional data into digital contents so that the distortion by embedding them is imperceptible, where the additional data are called watermark. One of the applications of digital watermarking is tamper detection and recovery. The digital watermarking methods for tamper detection and recovery can detect the tampered region in a tampered image and recover it. In this application, the digital watermarking methods solve the problem that digital contents can be tampered so that the falsication is not detected. In this manuscript, we propose a new method for tamper detection and recovery of JPEG images by using a RS code. The proposed method embeds the check symbols corresponding to an information-reduced original image into the LSBs of the quantized DCT coefcients in a JPEG image directly. Therefore the method completely extracts the embedded data from watermarked JPEG images. We investigated the performance of the proposed method in view of the image quality and the ability of tamper detection and recovery. The experiments of the image quality conrmed the imperceptibility of watermarked images and images used for recovery. The experiments of the ability of tamper detection and recovery conrmed the maximum size of recoverable tampered region and the difculty of undetectable falsication.

I. I NTRODUCTION Digital watermarking is a technique that embeds additional data into digital contents so that the distortion by embedding them is imperceptible, where the additional data are called watermark. One of the applications of digital watermarking is tamper detection and recovery [1]-[7]. The digital watermarking methods for tamper detection and recovery can detect the tampered region in a tampered image and recover it. In this application, the digital watermarking methods solve the problem that digital contents can be tampered so that the falsication is not detected. Minami, et al. [1] proposed the construction of recoverable BMP image data based on error correcting codes. Fridrich, et al. [2] proposed the method which embedded an image into itself based on the reduction of the amount of embedded data by using the quantization of the DCT coefcients of the image. This method didnt have the robustness against JPEG compression with quality factor 85%. Lin, et al. [3][5] proposed the methods which embedded authentication bits and recovery bits into original images so that the methods had the robustness against JPEG lossy compression etc.. Therefore it was considered that these methods couldnt detect the falsication which was milder than JPEG lossy compression at all. Minami, et al. [6] proposed the method for tamper detection

where R, G, B , Y , Cb and Cr represent pixel values of R, G, B, Y, Cb and Cr components respectively.

9781424460175/10/$26.00 c 2010 IEEE 309

RGB image RGB to YCb Cr composition of MCU 2-D DCT quantization runlength coding Huffman coding JPEG image quantization table Huffman table

16 12 14 14 18 24 49 72

11 12 13 17 22 35 64 92

10 14 16 22 37 55 78 95

16 19 24 29 56 64 87 98

24 26 40 51 68 81 103 112

40 58 57 87 109 104 121 100

51 60 69 80 103 113 120 103

61 55 56 62 77 92 101 99

Fig. 2.

Example of quantization table.

E. Runlength coding and Huffman coding Quantized DCT coefcients are encoded with runlength coding and Huffman coding. This process is lossless compression. A JPEG coded sequence consists of Huffman codewords, Huffman table and the quantization table. III. P ROPOSED M ETHOD A. Generation of Watermark The watermark used in the proposed method consists of the check symbols of a Reed-Solomon code (RS code). The information symbols corresponding to the check symbols are the information-reduced sequences of the quantized DCT coefcients obtained from JPEG images. In this section, we describe the policy of composition of the information symbols in Sec. III-A1 and the generation of the information symbols in Sec. III-A2. Then we describe the generation of the check symbols from the obtained information symbols in Sec. III-A3 and the transform of the check symbols into a watermark in Sec. III-A4. 1) Policy of Composition of Information Symbols: We use m bits per symbol and n symbols per unit in ReedSolomon encoding. One information symbol is obtained from the quantized DCT coefcients in dense locations so as to decrease the number of error symbols by tamper. On the other hand, One unit is composed of information symbols in sparse locations so as to decrease the number of error symbols in a unit. We save the composition of information symbols in an embedding procedure, and use it in an extracting procedure. 2) Generation of Information Symbols: step1 We obtain the quantized DCT coefcients from an original JPEG image with Nx Ny pixels. The quantized DCT coefcient in the (u, v )-th block with 8 8 pixels is represented by Fuv (i, j ), where (i, j ) represents the coordinate of the coefcient in the block, and 0 i < 8, 0 j < 8, 0 u < Nx /8, and 0 v < Ny /8. step2 The LSBs of the quantized DCT coefcients which will be used for embedding are replaced with 0. In virtue of this, the information symbols are unchanged by embedding. For example, the circled coefcients in Fig. 4 are used for embedding in our experiments. step3 The direct current components of each block are
310

Fig. 1.

Flow of JPEG compression.

B. Composition of minimum coding units First Y, Cb and Cr components are respectively divided into blocks with 16 16 pixels. Next each block of Y components is divided into four blocks with 8 8 pixels. Blocks with 16 16 pixels of Cb and Cr components are down-sampled into blocks with 8 8 pixels. An MCU consists of four 8 8 blocks of Y component, one 8 8 block of Cb component and one 8 8 block of Cr component, which are obtained in the above process. C. 2-dimensional discrete cosine transform The 8 8 blocks obtained in Sec. II-B is transformed with 2-D DCT into 8 8 blocks of DCT coefcients. 2-D DCT is represented by the following equations: Suv = Cu Cv 4
1 , 2 7 7

sxy cos
x=0 y =0

(2x + 1)u (2y + 1)v cos , 16 16 (4) (5)

Ci =

when i = 0 when i = 0

1,

where Suv and sxy represent a DCT coefcient at the coordinate (u, v ) and a pixel value at the coordinate (x, y ) respectively. S00 is especially called direct current component and other Suv s are called alternating current components. Suv with larger u and v represents a high frequency component. D. Quantization of DCT coefcients DCT coefcients obtained in Sec. II-C are quantized for data reduction by an 8 8 quantization table. Fig. 2 shows the example of a quantization table. The upper left values in quantization table are small. This is because large changes of the upper left DCT coefcients representing low frequent components cause large degradation of image quality. On the other hand, the lower right values in the table are large. This is because changes of the lower right DCT coefcients representing high frequency components are hard to perceive.

modied by the following equation: Fuv (0, 0) = Fuv (0, 0), Fuv (0, 0) Fu(v1) (0, 0),

the last of ACb and ACr by the following equations: a Y f = when 0 f < LAY ed, aY f none when LAY ed f < LAY , aCb f aY (LA aCr f aY (L
ed)+(f LACb )

(7)

when v 0 mod 2, when v 1 mod 2, a Cb f = (6) a Cr f =

when 0 f < LACb , when LACb f < LACb + ed 2 , (8) when 0 f < LACr , when LACr f < LACr +
ed 2 ,

In virtue of this, the amount of the information bits of the direct current components is decreased. step4 The quantized DCT coefcients modied by step2 and step3 are quantized by 2p if they are used for the information symbols, where p represents the index of the coefcient in zig-zag scan order in a block. For example, the numbered coefcients in Fig. 4 are used for the information symbols in our experiments. Moreover it is modied so as to be in [2p 1 , 2p 1 1]. In virtue of this, the modied quantized DCT coefcient can be represented by p bits, where we use different values of p and p for Y component and Cb , Cr components based on the human visual system. step5 The secret function g is xed in advance. The input of g is the modied quantized DCT coefcients which are not used for information symbols. The output of g is a short bit sequence to be used as the part of the information symbols. The amount of the output of g is xed as 6 bits for Y component and 4 bits for Cb , Cr components. step6 The information symbols of each block are composed of the bit sequences obtained by step4 and step5. 3) Generation of Check Symbols: Firstly we randomly rearrange the obtained information symbols in every block. We use the psudo-random sequence with secret key key s for the rearrangement, where s represents the seed of the psudo-random sequence. Then we obtain the check symbols by RS-encoding the rearranged information symbols. The check symbols obtained from a color component c are represented c by Ac = {ac f |af {0, 1}, 0 f < LAc }, c {Y, Cb , Cr }, where LAc represents the length of the check symbols Ac . In this paper, we use LACb = LACr < LAY because of the importance of Y component for human visual system. 4) Transform Check Symbols into Watermark: The watermark embedded into a color component c is represented by c c Wc = {wkl (t) | wkl (t) {0, 1}, 0 k < Nx /8, 0 l < c Ny /8, 0 t < LWc }, where wkl (t) and LWc represent the tth watermark bit embedded into the (k, l)-th block in the color component c and the length of the watermark bits embedded into a block respectively. As described in Sec. III-A3, the length LAY of the check symbols for Y component is larger than LACb and LACr . Therefore the latter part of AY is embedded into Cb and Cr c by adding the latter part of AY to components. We obtain A
311

AY

ed 2 )+(f LACr )

(9)

where e and d respectively represent the number of bits which constitute one check symbol and the number of check symbols for Y component embedded into Cb or Cr components. Then c we decide watermark bits wkl (t) by the following equation:
c wkl ( t) =

a c (lNx /8+k)LW

c +t a c ((l+1)Nx /8k)LWc (t+1)

when l is even, when l is odd,

(10)

The watermark is embedded from left to right into even rows and from right to left into odd rows by Eq. (10). In virtue of this, the watermark bits which constitute one symbol are embedded into the same or neighboring blocks. B. Embedding and Extracting 1) Policy of Desision of Embedding Locations: Practical tampers modify not only one block but also some dense blocks. Then the number of error symbols in one codeword will increase enough to exceed the error correcting ability if the watermark corresponding to one codeword is embedded into the dense blocks. Therefore we embed the watermark corresponding to one codeword into sparse blocks. 2) Embedding Procedure: step1 We obtain the quantized DCT coefcients from an original JPEG image with Nx Ny pixels. The quantized DCT coefcient for embedding in the (k, l)-th block with 8 8 pixels is represented by c Dkl (t), where t corresponds to the watermark bit c wkl (t) embedded into Dkl (t), and 0 t < LWc , 0 k < Nx /8, and 0 l < Ny /8. c c step2 The LSB of Dkl (t) is replaced with wkl (t) for all k , l, t, and c. step3 We obtain the watermarked JPEG image by Huffman-encoding the watermarked quantized DCT coefcients and writing the Huffman table in the header. 3) Extracting Procedure: c ( t) step1 We obtain the quantized DCT coefcients D kl from a tampered JPEG image with Nx Ny pixels in the same manner as the embedding procedure. c step2 We obtain the watermark bit w kl (t) from the LSB of c Dkl (t) for all k , l, t, and c.

C. Tamper Detection and Recovery For tamper detection, the watermarks Wc are rstly extracted from a suspicious JPEG image. Next they are transformed into the check symbols Ac by the inverse transform of Eq. (10), while the information symbols are calculated from the suspicious JPEG image by the same procedure as Sec. III-A2. We can detect tampering by RS-decoding the check symbols and the information symbols. There is tampering in the suspicious JPEG image if some errors are detected and then we try to recover the tampering as the following procedures, where only the existence of tampering is reported if the tampering is out of the ability of the error correction. For recovery, we rstly correct the errors of the information symbols by RS-decoding and obtain the information-reduced sequences of the quantized DCT coefcients. Next we modify the information-reduced sequences so that the elements of them are in [2p 1 , 2p 1 1] and are inverse-quantized by 2p , where we modify the part of the information-reduced sequences corresponding to direct current components to add Fu(v1) (0, 0) when v 1 mod 2 in advance. Then we obtain the quantized DCT coefcients for recovery. Finally we recover the tampered parts by replacing the tampered quantized DCT coefcients with the quantized DCT coefcients for recovery. IV. E XPERIMENTS A. Environments Shown in Fig. 3, we used four color images aerial, girl, lena, and mandrill as original images. They were standard images widely used for experiments and were compressed by JPEG compression of quality factor 75% without the sampling for Cb and Cr components in gimp1 . The size of all original images was 256 256 pixels, that is Nx = 256 and Ny = 256. We used nine secret keys s = 100, 200, . . ., 900 for rearrangement in Sec. III-A3, the sum of inputs as the secret function g in Sec. III-A2, and d = 8 in Sec. III-A4. Figure 4 shows the parameters p and p for Y component and Cb , Cr components and the locations of the coefcients for embedding. In Fig. 4, the values represent the parameters p or p corresponding to its coordinate, while the circled coordinates represent the locations of the coefcients for embedding, where the coordinates corresponding to smaller values in quantization table are selected as the locations for embedding. The decision of p and p was based on the tradeoff of the quality of recovery images and the ability of tamper detection. The parameter p affects both of the quality of recovery images and the length of information symbols. Therefore we rstly decided the sum of p , the amount of the output of g and the number of embedded bits in one block so as to control the tradeoff. Then we decided the individual p and p so that the quality of recovery image was high. We used the RS code of error correcting ability t = 20 on GF (210 ). The number m of the bits in one symbol was m = 10, and
1 http://www.gimp.org/

aerial

girl

lena
Fig. 3.

mandrill
Original images.

the number n of the symbols in one unit was n = 256. Then the number e of bits which constitute one check symbol was e = 2tm = 420. The number of embedded bits into Y, Cb , and Cr components were 8400, 6720, and 6720 bits respectively. Figure 5 shows the interval of the location for embedding watermarks, where the location for embedding is decided based on the description in Sec. III-B1. Shown in Fig. 5, we embedded one sequence of check symbols corresponding to one unit of information symbols into the blocks in shadowed locations. In other words, we embedded one sequence of check symbols into the set of the dense 2 2 blocks in every 8 8 blocks. On the other hand, the unit of information symbols was composed of the symbols which were obtained one by one from every two blocks in the row and the column as described in Sec. III-A1. We used PSNR for the evaluation of image quality. PSNR was calculated by the following equation. PSNR = 10 log10 1 MSE = 3Nx Ny 2552 , MSE
3 Nx Ny

(11) (12)

(orgi embi )2 ,
i=1

where orgi and embi represent the pixels in an original JPEG image and a watermarked JPEG image respectively. We used the PSNR of a recovery image for the evaluation of image quality of recovery, where the recovery image represents the image reconstructed by using the quantized DCT coefcients for recovery in Sec. III-C. B. Image Quality of Watermarked Images and Recovery Images Table I shows the PSNRs of the watermarked images and the recovery images, where the PSNRs of the watermarked images
312

8 7 7 6

7 7 5

6 5

6 4 4

4 4

TABLE I PSNR S OF WATERMARKED IMAGES AND RECOVERY IMAGES . watermarked images 35.74 35.72 35.66 35.73 recovery images 25.75 31.04 27.56 22.84 [dB]

aerial girl lena mandrill

p for Y component
0 0 0 0 0 0 0 1 0 0

p for Cb , Cr components
2 2 1 2 1 1

watermarked image p for Y component

Fig. 4.

recovery image

p for Cb , Cr components

Fig. 6.

Watermarked image and recovery image (lena).

Parameters for making watermark and coefcients for embedding.

8 8

TABLE II F ILE SIZES OF ORIGINAL IMAGES AND WATERMARKED IMAGES . original images 23,052 12,323 15,611 25,310 watermarked images 23,768 14,831 17,782 26,087 [byte]

aerial girl lena mandrill

the recovery image was like that of the image low-pass-ltered every block. C. File Sizes of Original Images and Watermarked Images
Fig. 5. Interval of locations for embedding.

are the averages of the images using nine secret keys. The maximum of the difference of the PSNRs by the secret keys was about 0.15 [dB], where the secret keys affect the PSNRs of not recovery images but watermarked images. Figure 6 shows the watermarked image using the secret key s = 500 and the recovery image of lena. Shown in Tab. I, the PSNRs of the watermarked images were around 35.7[dB] and independent of the original images. Therefore the image quality of the watermarked images was stable. This merit proceeded from the embedding procedure of the LSB replacement and the selection of the embedding locations corresponding to the smaller values in quantization table shown in Fig. 4. On the other hand, the PSNRs of the recovery images were dependent on the original images. The original images which consisted of noisy or texturelike regions tended to produce the recovery images with the low PSNRs. It is because of the information-reducing procedure in Sec. III-A2. Shown in Fig. 6, the degradation by embedding was hardly perceptible, while careful observation might discover the edges of block noises. The degradation of
313

The le sizes of watermarked images are often larger than those of original images when both of original images and watermarked images are compressed formats such as JPEG images. The increase of the le sizes of watermarked images should be slight. Table II shows the le sizes of the original images and the watermarked images. Shown in Tab. II, the differences between the le sizes of original images and watermarked images were relatively large when the le size of an original image was relatively small, while the differences were little when the le size of an original image was large. Therefore the increased le size of watermarked images would be allowed, although the control of them remains a future work. D. Performance of Tamper Detection and Recovery We investigated the performance of tamper detection and recovery by using three samples of tampered images of lena, where the performance theoretically depended not on an original image but on an error correcting ability and the number of modied symbols. Figure 7 shows the used tampered images and the recovered images corresponding to them. In Fig. 7(1), the size of the tampered part was 48 48 pixels. The modication in Fig. 7(1) was recovered

E. Security of Publicity of Algorithm The proposed method is protected from undetectable falsication by the secret information s and g . Therefore the secret informations should be managed strictly. The publicity of the locations for embedding enables unrecoverable falsication. However such the falsication can be detected. (1) 48 48 square tamper (1r) recovered 1 V. C ONCLUSION We have proposed a watermarking method for tamper detection and recovery of JPEG images. The proposed method detects a tampered part and recover it by a Reed-Solomon code. Moreover the proposed method can employ JPEG images as original images by using LSBs of the quantized DCT coefcients for embedding. The experiments have conrmed that the proposed method was able to detect a tampered part of size up to 48 48 pixels and recover it rightly. It was also conrmed that the tampered parts of which the size was larger than 48 48 pixels can be detected. Our future works should be to investigate the performance against sparse tampered parts and to consider an application to original images of larger size in view of the way to choose information symbols and to select the embedding locations. R EFERENCES
[1] N.Minami, K.Wakasugi, M.Kasahara, and S.Tazaki, A construction method of restorable photo copies and its evaluation, IEICE Trans. Inf. Syst. (Japanese Edition), vol.J81-D-II, no.11, pp.25352546, 1998. [2] J.Fridrich and M.Goljian, Images with self-correcting capabilities, IEEE Int. Conf. on Image Processing Proceedings ICIP99, pp.792796, 1999. [3] C.Y.Lin and S.F.Chang, Semi-fragile watermarking for authenticating JPEG visual content, Proc. SPIE Int. Conf. on Security and Watermarking of Multimedia Content II, vol.3971, pp.140151, Jan. 2000. [4] C.Y.Lin and S.F.Chang, SARI: Self-authentication-and-recovery image watermarking system, Proc. of the ninth ACM Int. Conf. on Multimedia, pp.628629, 2001. [5] C.Y.Lin, D.Sow, and S.F.Chang, Using self-authentication-and-recovery images for error concealment in wireless environments, Proc. SPIE Int. Conf. on Multimedia Systems and Applications IV, vol.4518, pp.267 274, Aug. 2001. [6] N.Minami and M.Kasahara, A new decoding method for digital watermark based on error correcting codes and cryptography, IEICE Trans. Fundamentals (Japanese Edition), vol.J87-A, no.7, pp.967975, 2002. [7] Y.Park, H.Kang, K.Yamaguchi, and K.Kobayashi, Watermarking for tamper detection and recovery, IEICE Electronics Express, vol.5, no.17, pp.689696, 2008. [8] H.Luo, Z.M.Lu, S.C.Chu, and J.S.Pan, Self Embedding Watermarking Scheme Using Halftone Image, IEICE Trans. Inf. & Syst., vol.E91-D, no.1, pp.148152, 2008. [9] Y.J.Chang, R.Z.Wang and J.C.Lin, A sharing-based fragile watermarking method for authentication and self-recovery of image tampering, EURASIP journal on advances in signal processing, vol. 2008, Article ID 846967, 17 pages, 2008. doi:10.1155/2008/846967 [10] F.J.MacWilliams and N.J.A.Sloane, The theory of error-correcting codes, North-Holland, Amsterdam, 1977. [11] K-work, JPEG - from concept to implementation by C++, Softbank Publishing, 1998. [12] S.Ono and J.Suzuki, Easy realization method for JPEG/MPEG2, Ohmsha, 1995.

(2) 60 60 square tamper

(2r) recovered 2

(3) three 28 28 squares tamper

Fig. 7.

(3r) recovered 3

Tampered images and recovered images(lena).

as shown in Fig. 7(1r), where the proposed method could theoretically recover a tampered part when the size of the tampered part was equal to or less than 48 48 pixels. In Fig. 7(2), the size of the tampered part was 60 60 pixels. The modication in Fig. 7(2) was not recovered as shown in Fig. 7(2r) because the size of tampered part was larger than 48 48 pixels. In Fig. 7(3), the size of the tampered parts was 28 28 pixels and then the number of the tampered parts was three, where the number of tampered pixels in Fig. 7(3) was approximately equalized to that in Fig. 7(1). The modication in Fig. 7(3) was not recovered as shown in Fig. 7(3r) because the sparse tampered parts affected the information symbols which belonged to one codeword and were embedded into sparse blocks. However the proposed method rightly detected all the locations of the tampered parts of the tampered images used in the experiments. In this experiment, we employ the sum of inputs as the secret function g in Sec. III-A2. Then the modication by the rearrangement among the coefcients in high frequency cannot be detected. Therefore it is important that the secret function g is not estimated.
314