A custom dataset was generated for the use of this project. To achieve this, we scraped an online database of humorous captioned images, known as memes, through their webpage called “meme generator”. To ensure the proper working and focused performance of the model, each meme format was retrieved individually and stored in a separate file for the model’s usage. The size of the datasets for each of the formats ranged from 350 entries to 20000 entries, allowing the observation of the model’s reactions to widely varying dataset sizes.
LSTM stands for long-short term memory. It is an RNN but overcomes its drawbacks (vanishing or exploding gradient problem). There are sigmoid and tanh activation functions used inside the layer. In the model built in the project, we use ReLU to connect the LSTM layer to other layers. 8 In this model, we performed tokenization, vocabulary for characters. Modules: Keras, numpy and tensorflow Level of prediction: character level Model architecture: Model: "sequential"
lstm (LSTM) (None, 100, 256) 264192
dropout (Dropout) (None, 100, 256) 0
lstm_1 (LSTM) (None, 256) 525312
dropout_1 (Dropout) (None, 256) 0
Total params: 801,326 Trainable params: 801,326 9 Non-trainable params: 0
BiLSTM is just 2 LSTM layers but one in the forward direction and the other in the backward direction. This approach increases the sheer volume of data available to the model and increases the context availability In this model, we performed tokenization, padding and vocabulary for words(word embedding). Modules:keras, tensorflow, matplotlib, PIL Level of prediction: word level Model architecture: Model: "sequential"
embedding (Embedding) (None, 49, 100) 295500
10 bidirectional (Bidirectional (None, 49, 300) 301200
dropout (Dropout) (None, 49, 300) 0
lstm_1 (LSTM) (None, 100) 160400
dense (Dense) (None, 1477) 149177
Total params: 5,273,767 Trainable params: 5,273,767 Non-trainable params: 0
GPT 2 is a large-scale unsupervised language model which generates coherent sentences about arbitrary topics. This model can be fine-tuned to specific tasks. OpenAI developed GPT 2 with 40GB of text data and released a sample version as open source for researchers and academics to experiment. GPT 2 is a generative transformer model.
Small model: less number of lstm layer Large model: more number of lstm layers
No of characters generated: 100 Model: Small model No. of epochs trained: 20 epochs.
No of characters generated: 100 Model: Small No of epochs trained: 50 epochs
No of characters generated: 100 Model: Large No of epochs: 100
No of characters generated: 100 Model: large No of epochs: 100
Drawback: the model is overfitted as the generated text already exists in the dataset.
Textbooks y u no have CTRL-F Seed = flavor
through morning shows truck morning
the model is underfitted as the generated text shows little variance with different seeds.
Adding captions to images allows the images to target a wider range of audiences, and can be used by multiple companies to use a single image to construct varying storylines. The project yields a conclusion that while the datasets can produce working results, the improvements in accuracy and quality of the output generated, such as the captions being more humorous and meaningful, when using datasets of increasing size denotes that the performance of the model is directly proportional to the size of the dataset to some extent.