TensorFlow Courses Study Notes

TensorFlow Courses Study Notes

Study Notes for TF certificate and Advanced TF techniques.


TensorFlow in Practice Courses

  1. Intro to TensorFlow
  2. Convolutional Neural Networks
  3. Natural Language Processing
  4. Sequences, Time Series and Prediction

Details in Code


Advanced TensorFlow Courses

TensorFlow: Advanced Techniques Specialization

  1. Custom Models, Layers, and Loss Functions with TensorFlow
    1. Functional APIs
    2. Custom Loss Functions
    3. Custom Layers
    4. Custom Models
    5. Callbacks
  2. Custom and Distributed Training with TensorFlow
    1. Differentiation and Gradients
    2. Custom Training
    3. Graph Mode
    4. Distributed Training
  3. Advanced Computer Vision with TensorFlow
    1. Introduction to Computer Vision
    2. Object Detection
    3. Image Segmentation
    4. Visualization and Interpretability
  4. Generative Deep Learning with TensorFlow
    1. Style Transfer
    2. AutoEncoders
    3. Variational AutoEncoders (VAEs)
    4. GANs

Part 1&2: Custom and Distributed Strategies

Part 3: Advanced Computer Vision

C3W1: Introduction to Computer Vision

  • Image Classification

    • multi-class classification (cat, dog, horse …)
      • binary classification (subset of the problem) (is cat? false or true)
    • multi-label classification (cat and dog in one image)
  • Object Detection

    • for each object:
      • confidence scores
      • bounding boxes
    • popular algorithms:
      • R-CNN, Faster-RCNN, YOLO, SSD
  • Image Segmentation

    • Semantic Segmentation:
      • all objects of the same type for a single classification.
      • same class -> one segment
      • each pixel is associated with one class
      • popular models: FCN (fully conv nn), U-Net, DeepLab
    • Instance Segmantation:
      • each instance of a person is identified as a separate segment.
      • popular models: Mask R-CNN
  • Transfer Learning

    • Pre-trained model features
  • Object Localization and Detection

    • Architecture: after Flatten+Dense -> 2 output of network (below)
      • regression: bounding box (for object localization)
      • n-way softmax: class (for classification)
    • Intersection Over Union (IoU)
    • Non-Maximum Suppression (NMS)

C3W2: Object Detection

  • Sliding windows

    • NMS: one of them might be the best fit as the bounding box
      • likely has the most overlap with the true label of the bounding box
      • selecting the highest of IoU of bonding box.
  • Two Stages to Obejct Detection

    • Region proposal
    • Object detection and classification
  • R-CNN: regions with CNN features

    • 1 input image ->
    • 2 extract region proposals (~2k) -> (warped region) ->
      • each region proposal is a grouping of smaller segments using the selective search method above.
    • 3 compute CNN features ->
      • using pre-train cnn, AlexNet architecture (input fitted by warped region step)
    • 4 classify regions
    • Main processes:
      • get final region proposals
      • extracted them from the image, warped size
      • fed into ConvNet (e.g. AlexNet) for feature extraction, then get 2 outputs:
        • fed into a classification layer (SVM) for getting label of the image
        • fed into a regression to get the BBox for the areas of interest
    • Disadvantages:
      • slow, memory
  • Transfer Learning for R-CNN

    • Pre-train: auxiliary task
      • large auxiliary dataset
    • Fine-tune: domain specific task
      • warped region proposals

  • Fast R-CNN
    • solve by removal the expensive Selective Search Algorithm
  • Faster R-CNN

C3W3: Image Segmentation

C3W4: Visualization and Interpretability

Part 4: Generative Deep Learning

C4W1: Style Transfer

  • Approaches to Styple Transfer
    • Supervised Learning
      • Pairs: original & stylized images (Neet a lot of pairs.)
    • Neural Style Transfter
      • Pre-trained model
      • Inputs: a single pair of images
        • Extract Style from image 1
        • Extract Content from image 2
      • Generated image to match style and content
        • In a loop: minimize loss
      • Main Processes:
        • Content Image -> pre-trained CNN (VGG-19) -> Content Loss (deep layers) with Generated Image
        • Style Image -> pre-trained CNN (VGG-19) -> Style Loss (in any Conv layers) with Generated Image
        • Generated Image (looks like content image at beginning)
        • Total Loss = w1 * Content Loss (ori img, gen img) + w2 * Style Loss (sty img, gen img)
          • Content Loss: element-wise subtraction, element-wise squire, reduce sum, with weight (*0.5)
          • Style Loss: Gram Matrix (style feature representation)
          • Einstein sum, numpy.einsum for Gram Matrix calculation
          • Solution for introduce noise: (using variation loss)
            • Decrease high frequency artefacts
            • Explicit regularization term on high frequency components
        • Update input (generated) image with gradients wrt. Loss using optimizer.
      • Advantages:
        • Requires only single pair of content image and style image.
      • Drawbacks:
        • Requires many passes for stylizing each image.
        • Tacks time in stylizing every time an input is given.
      • Paper: Perceptual Losses for Real-Time Style Transfer and Super Resolution
    • Fast Neural Style Transfer
1
2
3
4
5
6
7
8
9
# with total variantion loss
def calculate_gradient(image, content_targets, style_targets,
style_weight, content_weight, with_regularization=False):
total_variantion_weight = 30
with tf.GradientTape() as tape:
if with_regularization:
loss += total_variation_weight * tf.image.total_variation(image)
gradients = tape.gradient(loss, image)
return gradients

C4W2: AutoEncoders

  • Deep Auto-Encoder (Stacked Auto-Encoders):
    • input -> encoder -> bottleneck -> decoder -> reconstructed
  • Convolutional Auto-Encoders:
    • input -> conv2d_1 -> conv2d_2 -> bottleneck -> upsampling_1 -> upsampling_2 -> conv2d -> reconstructed
    • better quality of reconstructed image than deep AE.
  • Denoising with an Auto-Encoder:
    • if input with noise
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
## deep auto-encoder
def deep_autoencoder():
encoder = Dense(128, 'relu')(inputs)
encoder = Dense(64, 'relu')(encoder)
bottleneck = Dense(32, 'relu')(encoder)
decoder = Dense(64, 'relu')(bottleneck)
decoder = Dense(128, 'relu')(decoder)
output = Dense(784, 'sigmoid')(decoder)
return bottleneck, output

## conv auto-encoder
# padding='same'
def conv_encoder(inputs):
conv1 = Conv2d(64, (3,3), 'relu')(inputs)
maxp1 = MaxPooling2D(pool_size=(2,2))(conv1)
conv2 = Conv2d(128, (3,3), 'relu')(maxp1)
maxp2 = MaxPooling2D(pool_size=(2,2))(conv2)
return maxp2

def bottle_neck(inputs):
btnk = Conv2d(256, (3,3), 'relu')(inputs)
visualization = Conv2d(filters=1, kernel_size=(3,3), 'sigmoid')(btnk)
return btnk, visualization

def conv_decoder(inputs):
conv1 = Conv2D(128, (3,3), 'relu')(inputs)
upsp1 = UpSampling2D(size=(2,2))(conv1)
conv2 = Conv2D(64, (3,3), 'relu')(upsp1)
upsp2 = UpSampling2D(size=(2,2))(conv2)
output = Conv2D(filters=1, kernel_size=(3,3), 'sigmoid')(upsp2)
return output

C4W3: Variational AutoEncoders (VAE)

C4W4: GANs

  • GAN Architecture:

    • noise -> Generator (trainable model) -> fake data -> Discriminator (trainable model) <- real data
    • activation: selu (not relu)
    • loss: binary crossentropy
    • paper Self-Normalizing Neural Networks
  • Training Phases:

    • train discriminator
      • train by fake and real data
      • learn difference between them
    • train generator
      • produce a fake images (trick discriminator)
      • discriminator parameters frozen in this step
      • discriminator para will pass to generator
  • Issue: mode collaps

    • beacuse of adversarial relationship for both G and D want to win.
  • Deep GANs (DCGANs):

    • Using Conv to improve GANs performance
    • Generator:
      • use Conv2DTranspose instead of Conv2D + Pooling
      • all layers’ activation ‘selu’, except output layer tanh
    • Discriminator:
      • use strides=2 instead of Pooling after Conv2D
      • activation LeakyReLU use for Conv2D

Details in Code


Preparation

Other Linkes

TensorFlow Courses Study Notes

https://evawyf.com/2021/08/17/tf-courses/

Author

Eva W.

Posted on

2021-08-17

Updated on

2021-08-17

Licensed under

Comments