The main new thing in the version in that it allows the users to specify via the command line how many words should be considered when generating the next one. The bigger the step number the closer the generated text is to the original one. The value used in mctext-0.1 was 2, and this is also the default in this one. The number of steps can be set using the --steps command line switch.

In this version also couple of bugs were fixed (mostly segmentation faults). Another change the regular user will not notice, as it happened under the hood. I’ve redesign the program, and gave better software architecture that hopefully will allow one to extend its abilities and generalize its output generation.

I planned to add more features in this release but due to lack of time, I’ve decided to release as-is. The design rewrite is part of a future plan to allow mctext to operate on music pieces (probably MIDI). This should add a new dimension to the program and allow it to utilize the new generalized structure to generate new music pieces based on sample ones. This future project can have much nicer results that just plain text generation. I really hope I’ll find the time to complete it.

The new package can be downloaded found here – mctext-0.2.tar.bz2. Compilation and installation remained the same as in the previous version. The only dependency is the Boost C++ library.

This is a free software, so please fill free to modify or hack it any way you like. It would be great if you can send a comment when you do so. Also, if you got an interesting idea how this program can be used or modified, please comment.

How does it work?

mctext reads the given file and treats it as a list of words. Now it randomly chooses two adjacent words and puts in the output string. Now the text generation employs Markov Chains to continue. It takes the last two words in the output string, and searches for all the words that follow them in the sample file. He choose between those words randomly and adds the chosen one to the string. After doing so, it repeats the process until enough new text is generated.

For example, this was an output of the program when given 500 posts from Tech Crunch as sample text:

$ ./mctext -w 100 tc.txt
declined to name a specific position on the internet is now 
extended through the birth of high velocity P2P file sharing
and broadcasting short experiences, thoughts and fantasies.
By that we can look forward to seeing everyone. Loic Le Meur,
a well of useful contextual information that would be complete
without a second's hesitation. DonorsChoose Doing Well, But
Fred Wilson and the

(tc.txt was the file holding the text of the 500 posts)

Compiling and Using mctext

If you want to try it yourself, download the source package from here. Compiling is pretty straight forward (./configure && make). The program depends on the Boost library. Some Linux distributions separate the additional Boost libraries from the core ones, so if it’s your case you will need to install the program-options library.

Invoking the program is simple. Just pass it the sample text file as argument and use -w NUM flag to specify how much words do you want it to generate. mctext can also take the sample text from stdin. See mctext --help for more information.

mctext is a new project, and the current implementation was a proof-of-concept. As such, there is still a lot to improve and look up to.For the next I’m planning to allow changing the number of words considered at each step from the command line, as well as improving the sentence recognition. If you found a bug, or you have any suggestion for new feature I will be glad to hear.

Update: I’ve released a new version of mctext.