Made changes in structure and repo

kad99kev · Jul 5, 2021 · 1ee2c0d · 1ee2c0d
1 parent e7c6769
commit 1ee2c0d
Show file tree

Hide file tree

Showing 7 changed files with 24 additions and 63 deletions.
diff --git a/.DS_Store b/.DS_Store
diff --git a/MNIST-GANs/ACGAN/README.md b/MNIST-GANs/ACGAN/README.md
@@ -1,4 +1,4 @@
-# Auxillary Classifier Generative Adversarial Network (cGAN)
+# Auxillary Classifier Generative Adversarial Network (ACGAN)
 
 ### What are ACGANs?
 The ACGAN like any normal GAN consists of a generator and a discriminator. However, in ACGANs, every generated sample has a corresponding class label (that belongs to the available classes) in addition to the noise. This class label helps the model to generate images based on the label given.

diff --git a/MNIST-GANs/CGAN/DigitMNIST-CGAN.ipynb b/MNIST-GANs/CGAN/DigitMNIST-CGAN.ipynb
diff --git a/MNIST-GANs/DCGAN/FashionMNIST-DCGAN.ipynb b/MNIST-GANs/DCGAN/FashionMNIST-DCGAN.ipynb
diff --git a/MNIST-GANs/GAN/DigitMNIST-GAN.ipynb b/MNIST-GANs/GAN/DigitMNIST-GAN.ipynb
diff --git a/README.md b/README.md
@@ -1,80 +1,40 @@
 # Face Generation from Textual Description using Generative Adverserial Networks 📝  2️⃣ 👧👱 [![Open in Streamlit](https://static.streamlit.io/badges/streamlit_badge_black_white.svg)](https://share.streamlit.io/ELITA04/FGTD-Streamlit/app.py/)
 
+*Under Construction!*
 
-🔹 [Motivation](#motivation)
+---
 
-🔹 [Progress Ladder](#progress-ladder)
+🔹 [Abstract](#abstract)
+
+🔹 [Notebooks](#progress-ladder)
 
 🔹 [Results](#results)
 
 🔹 [Future Scope](#future-scope)
 
 🔹 [Reference Papers](#reference-papers)
 
-## Motivation  
-
-Powerful Generative Adverserial Networks have been used in the past to automatically synthesize realistic images from text. However, these existing tasks have been used for simpler tasks such as flowers and birds.
-So, our aim is to focus on a less addressed domain of face generation from fine-granined textual description of faces.
-
-As we are still undergrad students, we decided to not only build our main GAN, but also a ladder of GANs that helped strengthen our understanding, in hopes to inspire others.
-
-## Progress Ladder 
-
-### Step 1. We Started With Simple GANs On The MNIST Digit Dataset🔢
-
-🔗 [Code, References and Output](https://github.com/ELITA04/Face-Generator/tree/master/MNIST-GANs/GAN)
+---
 
-### Step 2. Progressed Our Way Up To Understanding Other GAN Architectures On The Digit and Fashion MNIST Datasets. 👗 👕
+## Abstract  
 
-🔗 [Deep Convolution GANS - Code, References and Output](https://github.com/ELITA04/Face-Generator/tree/master/MNIST-GANs/DCGAN)
+Majority of current text-to-image generation tasks are limited to creating images like flowers (Oxford 102 Flower), birds (CUB-200-2011), and Common Objects (COCO) from captions. The existing face datasets such as Labeled Faces in the Wild and MegaFace lack description while datasets like CelebA have attributes associated but do not provide feature descriptions. Thus, in this paper we build upon an existing algorithm to create captions with the attributes provided in the CelebA dataset, which can not only generate one caption but it can also be extended to generate N captions per image. We utilise Sentence BERT to encode these descriptions into sentence embeddings. We then perform a comparative study of three models - DCGAN, SAGAN and DFGAN, by using these sentence embeddings along with a latent noise as the inputs to the different architectures. Finally, we calculate the Inception Scores and the FID values to compare the output images across different architectures.
 
-🔗 [Conditional GANS - Code, References and Output](https://github.com/ELITA04/Face-Generator/tree/master/MNIST-GANs/CGAN)
+---
 
-🔗 [Auxillary Conditional GANS - Code, References and Output](https://github.com/ELITA04/Face-Generator/tree/master/MNIST-GANs/ACGAN)
+## Notebooks
 
-We documented our losses and make a report of our learnings using **Weights & Biases ✨**. The report can be found [here](https://wandb.ai/kad99kev/mnist-gans/reports/MNIST-GANs--VmlldzoyMTE4NzE).
+### MNIST
+| Model        | Colab Link     |
+| ------------- |-------------|
+| [Vanilla GAN](MNIST-GANs/GAN)      | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1setqmENPRRriznB8j2XL55RjSkMfMWCf?usp=sharing) |
+| [DCGAN](MNIST-GANs/DCGAN)     | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1cBsnZTL0bp7o9lfBez1FyedDxo2Yf6wh?usp=sharing)      |
+| [CGAN](MNIST-GANs/CGAN) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1X7xD1sX3iJggqMuDvnn7EtIUIsc5GDDn?usp=sharing)      |
+| [ACGAN](MNIST-GANs/ACGAN) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1-OGeMxST6jFvSc5cq_Oc_-9ItqJ86taP?usp=sharing)      |
 
-### Step 3. Researched About Past Implementations On The Topic
-(Some of the research paper links)
+### Face
 
-* [FTGAN : A fully Generative Adverserial Network for Text to Face Generation](https://arxiv.org/pdf/1904.05729.pdf)
-* [Text2FaceGAN : Face Generation from Fine Grained Description](https://arxiv.org/pdf/1911.11378.pdf) 
-* [Generative Adverserial Text to Image Synthesis](https://arxiv.org/pdf/1605.05396.pdf)
-
-
-### Step 4. Created A Meaningful Text Dataset Using Celeb-A
-
-Here, the challenege faced was to create meaningful sentences using a mutli-labelled dataset. We considered the following 6 cateogeries:
-
-  🔸 The structure of the face
-
-  🔸 The facical hairstyle the person sports
-
-  🔸 The description of other facical features
-
-  🔸 The hairstyle of the person
-
-  🔸 Attributes that enhance his appearance
-
-  🔸 Accessories worn (if any)
-
-  An example of one of the sentences from the dataset
-   <img src = "assets/dataset.png">
-
-  ### Step 5. Created A Pipeline To Try Out Different Architectures
-  P.S We've added some of our notebooks. While our work is far from perfect we'd appreciate if you check them out and give valuable feedback.
-
-  :white_small_square: [DCGANS with Skip thought embeddings](https://colab.research.google.com/drive/1JDNojILmxeF5nUkYeNe6xank2gfvVML6?usp=sharing)
-
-  :white_small_square: [DCGANS with BERT embbeddings](https://colab.research.google.com/drive/1J4tdVVluwmij9jPvtKTaBd7cQZp6FDr1?usp=sharing)
-
-  :white_small_square: [HDGANS with BERT embeddings](https://colab.research.google.com/drive/1dWGRdtlqrvCt9OUX8Y1f0SjjHEIxI2Uo?usp=sharing)
-
-
-### Step 6. Created A Streamlit Application To Showcase Our Work
-Check it out [here](http://www.ekkm.xyz/)
-
-The code for the same can be found [here](https://github.com/kad99kev/FGTD-Streamlit)
+---
 
 ## Results
  <img src = "assets/result.png" >
@@ -86,6 +46,8 @@ The code for the same can be found [here](https://github.com/kad99kev/FGTD-Strea
 
 3. We’ve only maintained a wandb report for the initial MNIST GANs which we mentioned earlier. Nevertheless, we also have plans to make one for our Face GANs.
 
+---
+
 ## Reference Papers
 
 * [FTGAN : A fully Generative Adverserial Network for Text to Face Generation](https://arxiv.org/pdf/1904.05729.pdf)

diff --git a/requirements.txt b/requirements.txt
@@ -0,0 +1,2 @@
+sentence-transformers==1.1.0
+wandb==0.10.27