Generative Machine Learning

What would your computer say if it could talk? Well using generative machine learning we’re able to get a glimpse of what that might be like. Unlike other fields of machine learning, generative ML is still relatively un-explored. The main reason is that it doesn’t really solve problems the same way other fields of ML, such as classification, do so large corporations aren’t willing to pour tons of money into it. However, that doesn’t mean it isn’t an interesting way to learn about machine learning.

This project is going to revolve around the “char-rnn-tensorflow” github page created by Sherjilozair. This repository uses Multi-layer Recurrent Neural Networks (LSTM, RNN) for character-level language generation. It comes with some sample Shakespeare text that you can use to train the model and have it create fake but impressively real sounding text. Alright enough talk, lets jump in.

• First off we need to make a new file directory that we can clone the git repository to. So create a folder, navigate to it in the terminal/command prompt, then type git clone https://github.com/sherjilozair/char-rnn-tensorflow.git
• Sweet, now lets take a look at what we got. In the char-rnn folder you should see a number of files and folders. Open up the train.py file. It should look like this:



• These are all the possible command-line arguments you can pass when you are training it on some text. The default text is the tinyshakespeare file which contains about 1Mb of text from various Shakespeare plays, however you can pass it any text you like. It has parameters for training checkpoints; tensorboard so you can view the progress; the size of the neural network; the number of layers; number of epochs, sequence length, batch size, learning-rate, decay-rate, etc..

Feel free to play around with the parameters, however I recommend keeping the last 6 or 7 parameters unchanged until you are very comfortable with what you are doing.

• So once you’ve run train.py (it can take a while depending on your computer hardware), go ahead and run the sample.py file. Assuming you trained it on the default text you should get something like this:

 thanas thoursant here in slipf!
Could his skill deposed and suppress'd
Become her good maid eye, but so himself'd
You have every wants be coavere tears;
And here was collent me to seal my words.
Wither, he's some pardon smooth yours.

Gay:
My Lord Adulo, not time for that make thest;
Being you.

Second Keeper Gloush extremes that in death.

Gaim:
I pray you to my trial?

Clown:
Aim, you sleep, deny as if I was you:
My name by like itself to their mouths and thee.

PETRUCHIO:
Into trial.

NORFOLTO:
Come.

BRAKENBURY:
What sha few's prison, on them Secretio Jolces,
I'll swear up to her king him to a day.

• It looks weird but remember we trained it on text that was written in old-english. So I think it did pretty well. Also it creates fake names lol. Here is a sample of the original text for comparison:

First Citizen:
If I must not, I need not be barren of accusations;
he hath faults, with surplus, to tire in repetition.
What shouts are these? The other side o' the city
is risen: why stay we prating here? to the Capitol!

All:
Come, come.

First Citizen:
Soft! who comes here?

Second Citizen:
Worthy Menenius Agrippa; one that hath always loved
the people.

First Citizen:
He's one honest enough: would all the rest were so!

MENENIUS:
What work's, my countrymen, in hand? where go you
With bats and clubs? The matter? speak, I pray you.

• Anyway, I’m tired so I’m gonna wrap this one up. But I plan on writing a follow up post about some other things you can do with generative learning, so look out for that.
  Thanks!!