Clear Filters
Clear Filters

GAN intermediate layer for image multiplication

1 view (last 30 days)
So I have created a GAN that uses a regular noise input for image synthesis. However, I want to multiply or get the product of the generated image with an outside image to speed up training/ training convergence. Here is the basic idea:
%This is the layer setup I use for my GAN
numFilters = 128;
numLatentInputs = 256;
inputSize = [128 128 3];
projectionSize = [4 4 3];
layersGenerator = [
featureInputLayer(numLatentInputs)
projectAndReshapeLayer(projectionSize)
resize2dLayer('OutputSize',[4 4],'Method','bilinear')
batchNormalizationLayer
leakyReluLayer
resize2dLayer('OutputSize',[16 16],'Method','bilinear')
batchNormalizationLayer
leakyReluLayer
resize2dLayer('OutputSize',[32 32],'Method','bilinear')
batchNormalizationLayer
leakyReluLayer
resize2dLayer('OutputSize',[64 64],'Method','bilinear')
batchNormalizationLayer
leakyReluLayer
resize2dLayer('OutputSize',[128 128],'Method','bilinear')
batchNormalizationLayer
tanhLayer];
layersDiscriminator = [
imageInputLayer(inputSize,Normalization="none")
dropoutLayer(dropoutProb)
resize2dLayer('OutputSize',[128 128],'Method','bilinear')
convolution2dLayer(filterSize,numFilters,Stride=2,Padding="same")
leakyReluLayer(scale)
resize2dLayer('OutputSize',[64 64],'Method','bilinear')
convolution2dLayer(filterSize,2*numFilters,Stride=2,Padding="same")
batchNormalizationLayer
leakyReluLayer(scale)
resize2dLayer('OutputSize',[32 32],'Method','bilinear')
convolution2dLayer(filterSize,4*numFilters,Stride=2,Padding="same")
batchNormalizationLayer
leakyReluLayer(scale)
resize2dLayer('OutputSize',[16 16],'Method','bilinear')
convolution2dLayer(filterSize,8*numFilters,Stride=2,Padding="same")
batchNormalizationLayer
leakyReluLayer(scale)
resize2dLayer('OutputSize',[8 8],'Method','bilinear')
convolution2dLayer(8,1)
sigmoidLayer];
numEpochs = 2000;
miniBatchSize = 16;
learnRate = 0.0003;
gradientDecayFactor = 0.5;
squaredGradientDecayFactor = 0.999;
flipProb = 0.75;
validationFrequency = 5;
%The custom layer that I have already made:
classdef imageProduct < nnet.layer.Layer
properties (Constant)
ImageinputT = imread("image.jpg");
end
methods
function layer = imageProduct()
layer.Type = "Image Product";
layer.Description = "Combine Images";
layer.OutputNames = "out";
end
function [Z] = predict(~,X)
%gets the image size so the input image can be resized. Basically the target scale
[WidthX, HeightX] = size(X);
%resizes the imported image
GinputIm_scaled = imresize(crossAttention.ImageinputT, [WidthX HeightX]);
%converts images to double
GinputIm_scaled_single = single(GinputIm_scaled);
Xsingle = single(X);
%Gets the tensor product of the two images
Xnew = tensorprod(GinputIm_scaled_single, Xsingle);
Z = Xnew; %output image
end
end
end
  1 Comment
Rahul
Rahul on 1 Mar 2023
Edited: Rahul on 1 Mar 2023
What is exactly the issue? Are you receiving any error? Please post the error message if any.

Sign in to comment.

Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!