Assessing the perceptual quality of video is critical for monitoring and optimizing video processing pipelines. In this paper, we focus on predicting the perceptual quality for videos distorted by noise. Existing video quality metrics are generally focus on ``white", i.e., spatially un-correlated noise. However, white noise is very rare in realistic videos. Based on our analysis of the noise correlation patterns on a large and comprehensive video set, we build a video database that simulates the commonly encountered noise patterns. Using the database, we develop a perceptual quality metric that explicitly incorporates the noise pattern in quality prediction. Experimental results show that the proposed algorithm presents very high correlation with the perceptual quality of noisy videos.