what is colorization?


Colorizing black and white images has become an interesting art or as well as real-word application. So that, It always remains a curiosity by viewing at an old image that how a person looks and which color he is worn? The scene visualization is not only looking boring in black and white but colors also give extra details in image. So that, Colorization of images allows us to observe the actual scene and provides a sense of relatability. However, it can admit that the exact colorization of originally captured image is not possible. So that, The illustrator uses his experience to colorize the paint color of an old black and white image. Also, There are various distinctions regarding skin color, and building colors, and mountains, eyes color, and hair etc and also It is a costly, time-consuming and hectic process. 

Image result for colorization of black and white images blog

Figure 1: Conversion from black and white Image into colored. Courtesy by: Topaz Lab

     Moreover, Advance techniques and technologies play a cardinal role in the improvement of the previous methods of colorization. So that, The research community are focusing on introducing artificial intelligence so that the model can be trained as human expert. Also, A face requires at least 20 layers of green, pink and blue shades. Also, The AI algorithms saves a lot of time and computation. And, In the computer vision domain, colorization revolves around as an active research. The most of advance technology in computer-vision includes machine learning and deep learning algorithms, which is subdomain of artificial intelligence. So that, We cannot say that these models are optimal solution to this problem, but they provide impressive results when working on some images.      

Image Representation: 

Actually, The representation of black and white images forms a block of pixels. So that, Each value represents corresponding brightness raging from 0 to 255. The representation of images are purely dependent on pixels.

  Figure 2: Black and White Image pixel block. Courtesy by: Medium

Actually, The color images are the combination of red,and blue and green primary colors. The different intensities and arrangement of these colors form other shades of colors. Also A color image is the composition of color and contrast shown as:

Colorization techniques

Particulary, The purpose of colorization is not to recover the old image but to create some possible and useful colorization. Also, Neural network allows to find these possibilities that links greyscale image to colored. So that, The image colorization using deep learning results effective results when compared to ground truth. So, In this article, the advance researches are discussed below.     

Cheng, Yang, and Sheng presented a neural network framework using YUV color space for the colorization technique. So that, They described it as a regression problem having error as the difference between actual and predicted value. Isola, Zhang and Efros describes this problem as classification. So that,The input of neural network is the grayscale image and output would be is a distribution over CIE lab color space.

Figure 3: Neural Network architecture for Colorization

Some of them combine image to image translation with adversarial networks for the conversion of greyscale image into colors,and recover image from its’ edge map, or conversion of daylight into nighttime etc.  Also, There are numerous great researches on the subject of colorization. 

Generative Adversarial Network (GAN)

Actually, Generative Adversarial Network (GAN) is the technique presented by Ian Goodfellow. So that, This method introduces a better implementation of colorization. Also, It consists of a generator and a discriminator. Generator is used to generate images, while on the other hand, discriminator verify the image. Here, the goal is to generate such images that discriminator couldn’t differentiate between generated and original image. Also, The loss of generator should be low, and the discriminator would be high. There is various combination of neural networks available as Avinash Hindupur presented a list on GitHub named by “The GAN Zoo”.  

Online Applications 

Actually, Algorithmia, Colourise.sg and DeOldify are some online tools available for automatic colorization. Therefore, They work on the principle of GAN. DeOldify also facilitate in case of videos. They provide very impressive results for outdoor scenes but not in case on indoor. Although, the results are satisfactory. 

Python Implementation of colorization bot using neural network

So that Here, I described one way to build a colorization neural net. And It can be dividing into three steps including:

The first step is to understand the image representation. so that, Colorization bot is initialized as 40-line neural network. There I not any magic but a series of implementation that is easy to understand. 

The second step is to generate a generalize network which is compatible to beta version. and, In the last step, a classifier is merged into neural network by using millions of images for training. 

1. Get images

X = []

for filename in os.listdir(‘/data/images/Train/’): X.append(img_to_array(load_img(‘/data/images/Train/’+filename)))

2. Load weights

inception = InceptionResNetV2(weights=None, include_top=True) inception.load_weights(‘/data/inception_resnet_v2_weights_tf_dim_ordering_tf_kernels.h5’)

3. Encoder

encoder_input = Input(shape=(256, 256, 1,)) encoder_output = Conv2D(64, (3,3), activation=’relu’, padding=’same’, strides=2)(encoder_input) encoder_output = Conv2D(128, (3,3), activation=’relu’, padding=’same’)(encoder_output) encoder_output = Conv2D(128, (3,3), activation=’relu’, padding=’same’, strides=2)(encoder_output) encoder_output = Conv2D(256, (3,3), activation=’relu’, padding=’same’)(encoder_output) encoder_output = Conv2D(256, (3,3), activation=’relu’, padding=’same’, strides=2)(encoder_output) encoder_output = Conv2D(512, (3,3), activation=’relu’, padding=’same’)(encoder_output) encoder_output = Conv2D(512, (3,3), activation=’relu’, padding=’same’)(encoder_output) encoder_output = Conv2D(256, (3,3), activation=’relu’, padding=’same’)(encoder_output)

4. Decoder

decoder_output = Conv2D(128, (3,3), activation=’relu’, padding=’same’)(fusion_output) decoder_output = UpSampling2D((2, 2))(decoder_output) decoder_output = Conv2D(64, (3,3), activation=’relu’, padding=’same’)(decoder_output) decoder_output = UpSampling2D((2, 2))(decoder_output) decoder_output = Conv2D(32, (3,3), activation=’relu’, padding=’same’)(decoder_output) decoder_output = Conv2D(16, (3,3), activation=’relu’, padding=’same’)(decoder_output) decoder_output = Conv2D(2, (3, 3), activation=’tanh’, padding=’same’)(decoder_output) decoder_output = UpSampling2D((2, 2))(decoder_output) model = Model(inputs=[encoder_input, embed_input], outputs=decoder_output)

5.Create embedding

def create_inception_embedding(grayscaled_rgb): grayscaled_rgb_resized = [] for i in grayscaled_rgb: i = resize(i, (299, 299, 3), mode=’constant’) grayscaled_rgb_resized.append(i) grayscaled_rgb_resized = np.array(grayscaled_rgb_resized) grayscaled_rgb_resized = preprocess_input(grayscaled_rgb_resized) with inception.graph.as_default(): embed = inception.predict(grayscaled_rgb_resized) return embed

6. Image transformation

datagen = ImageDataGenerator(shear_range=0.4, zoom_range=0.4, rotation_range=40, horizontal_flip=True)

7. Generate training data

batch_size = 20 def image_a_b_gen(batch_size): for batch in datagen.flow(Xtrain, batch_size=batch_size): grayscaled_rgb = gray2rgb(rgb2gray(batch)) embed = create_inception_embedding(grayscaled_rgb) lab_batch = rgb2lab(batch) X_batch = lab_batch[:,:,:,0] X_batch = X_batch.reshape(X_batch.shape+(1,)) Y_batch = lab_batch[:,:,:,1:] / 128 yield ([X_batch, create_inception_embedding(grayscaled_rgb)], Y_batch)

8. Train model      

tensorboard = TensorBoard(log_dir=”/output”) model.compile(optimizer=’adam’, loss=’mse’) model.fit_generator(image_a_b_gen(batch_size), callbacks=[tensorboard], epochs=1000, steps_per_epoch=20)

9. Make predictions 

color_me = [] for filename in os.listdir(‘../Test/’): color_me.append(img_to_array(load_img(‘../Test/’+filename))) color_me = np.array(color_me, dtype=float) colors_me = 1.0/255*color_mecolour_me =gray2rgb(rgb2gray(color_me)) color_me_embed = create_inception_embedding(color_me) color_me = rgb2lab(color_me)[:,:,:,0] color_me = color_me.reshape(color_me.shape+(1,))

10. Test model

output = model.predict([color_me, color_me_embed]) output = output * 128

11. Output colorizations

for i in range(len(output)): cur= np.zeros((256, 256, 3)) cur[:,:,0] = color_me[i][:,:,0] cur[:,:,1:] = output[i] imsave(“result/img_”+str(i)+”.png”, lab2rgb(cur))

The model provides images from the classifier that predicts a sense of what is present in the image. Hence, allowing the model to match an object illustration with colors.

You may also know: Digital Image Processing


Please enter your comment!
Please enter your name here

eleven + seven =