Author : K. Ushasree 1
Date of Publication :7th May 2016
Abstract: Video inpainting is the process of repairing missing regions (holes) in videos. Most automatic techniques are computationally intensive and unable to repair large holes. To tackle these challenges, a computationally-efficient algorithm that separately inpaint foreground objects and background is proposed. Using Dynamic Programming, foreground objects are holistically inpainted with object templates that minimize a sliding-window dissimilarity cost function. Static background are inpainted by adaptive background replacement and image inpainting.In this propose a new video inpainting method which applies to both static or free-moving camera videos. The method can be used for object removal, error concealment, and background reconstruction applications. To limit the computational time, a frame is inpainted by considering a small number of neighboring pictures which are grouped into a group of pictures (GoP). This drastically reduces the algorithm complexity and makes the approach well suited for near real-time video editing applications as well as for loss concealment applications. Experiments with several challenging video sequences show that the proposed method provides visually pleasing results for object removal, error concealment, and background reconstruction context.
Reference :
-
[1] C. Guillemot and O. Le Meur, “Image inpainting: Overview and recent advances,” IEEE Signal Process. Mag., vol. 31, no. 1, pp. 127–144, Jan. 2014.
[2] C. Barnes, E. Shechtman, A. Finkelstein, and D. B. Goldman, “PatchMatch: A randomized correspondence algorithm for structural image editing,” ACM Trans. Graph., vol. 28, no. 3, pp. 24:1–24:11, Jul. 2009.
[3] D. Glasner, S. Bagon, and M. Irani, “Super-resolution from a single image,” in Proc. IEEE 12th Int. Conf. Comput. Vis., Sep./Oct. 2009, pp. 349–356.
[4] J. G. Apostolopoulos, W.-T. Tan, and S. J. Wee, “Video streaming: Concepts, algorithms, and systems,” HP Lab. Palo Alto, Palo Alto, CA, USA, Tech. Rep. HPL2002-260, 2002.
[5] Y. Matsushita, E. Ofek, W. Ge, X. Tang, and H.-Y. Shum, “Full-frame video stabilization with motion inpainting,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 7, pp. 1150–1163, Jul. 2006.
[6] K. A. Patwardhan, G. Sapiro, and M. Bertalmio, “Video inpainting under constrained camera motion,” IEEE Trans. Image Process., vol. 16, no. 2, pp. 545–553, Feb. 2007.
[7] T. K. Shih, N. C. Tang, and J.-N. Hwang, “Exemplarbased video inpainting without ghost shadow artifacts by maintaining temporal continuity,” IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 3, pp. 347–360, Mar. 2009.
[8] T. K. Shih, N. C. Tan, J. C. Tsai, and H.-Y. Zhong, “Video falsifying by motion interpolation and inpainting,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2008, pp. 1–8.
[9] Y. Wexler, E. Shechtman, and M. Irani, “Space-time completion of video,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 3, pp. 463–476, Mar. 2007.
[10] A. Newson, A. Almansa, M. Fradet, Y. Gousseau, and P. Pérez, “Video inpainting of complex scenes,” SIAM J. Imag. Sci., vol. 7, no. 4, pp. 1993–2019, 2014.
[11] O. Whyte, J. Sivic, and A. Zisserman, “Get out of my picture! Internetbased inpainting,” in Proc. Brit. Mach. Vis. Conf., 2009, pp. 1–11.
[12] W.-Y. Lin, S. Liu, Y. Matsushita, T.-T. Ng, and L.-F. Cheong, “Smoothly varying affine stitching,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 345– 352.
[13] M. Granados, J. Tompkin, K. I. Kim, J. Kautz, and C. Theobalt, “Background inpainting for videos with dynamic objects and a freemoving camera,” in Proc. Eur. Conf. Comput. Vis., 2012, pp. 682–695.
[14] S. Liu, L. Yuan, P. Tan, and J. Sun, “Bundled camera paths for video stabilization,” ACM Trans. Graph., vol. 32, no. 4, pp. 78:1– 78:10, Jul. 2013. [15] X. Chen, Y. Shen, and Y. H. Yang, “Background estimation using graph cuts and inpainting,” in Proc. Graph. Inter., 2010, pp. 97–103.
[16] S. Cohen, “Background estimation as a labeling problem,” in Proc. 10th IEEE Int. Conf. Comput. Vis., Oct. 2005, pp. 1034–1041.
[17] A. Criminisi, P. Pérez, and K. Toyama, “Region filling and object removal by exemplar-based image inpainting,” IEEE Trans. Image Process.,vol. 13, no. 9, pp. 1200–1212, Sep. 2004
[18] P. Buyssens, M. Daisy, D. Tschumperlé, and O. Lézoray, “Exemplarbased inpainting: Technical review and new heuristics for better geometric reconstructions,” IEEE Trans. Image Process., vol. 24, no. 6, pp. 1809– 1824, Jun. 2015. [Online]. Available: https://hal.archivesouvertes. fr/hal-01147620
[19] O. Le Meur and C. Guillemot, “Super-resolutionbased inpainting,” in Proc. Eur. Conf. Comput. Vis., 2012, pp. 554–567.
[20] O. Le Meur, M. Ebdelli, and C. Guillemot, “Hierarchical superresolution- based inpainting,” IEEE Trans. Image Process., vol. 22, no. 10, pp. 3779–3790, Oct. 2013.
[21] Y. Hu and D. Rajan, “Hybrid shift map for video retargeting,” in ProcIEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 577–584.
[22] Y. Pritch, E. Kav-Venaki, and S. Peleg, “Shift-map image editing,” in Proc. IEEE 12th Int. Conf. Comput. Vis., Sep./Oct. 2009, pp. 151–158.
[23] V. Kwatra, A. Schödl, I. Essa, G. Turk, and A. Bobick, “Graphcut textures: Image and video synthesis using graph cuts,” in Proc. ACM Trans. Graph., 2003, pp. 277–286.
[24] V. Kolmogorov and R. Zabin, “What energy functions can be minimized via graph cuts?” IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 2, pp. 147–159, Feb. 2004.
[25] Y. Boykov, O. Veksler, and R. Zabih, “Fast approximate energy minimization via graph cuts,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 11, pp. 1222–1239, Nov. 2001