#!/usr/bin/python
import urllib
import urllib2
import re
import socket
import os
socket.setdefaulttimeout(30)
#adjust the site here
search_term="site:guyrutenberg.com"
def main():
    headers = {'User-Agent': 'Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4'}
    url = "http://www.google.com/search?q="+search_term
    regex_cache = re.compile(r'<a href="(http://\d*\.\d*\.\d*\.\d*/search\?q\=cache.*?)".*?>Cached</a>')
    regex_next = re.compile('<a href="([^"]*?)"><span id=nn></span>Next</a>')
 
    #this is the directory we will save files to
    try:
        os.mkdir('files')
    except:
        pass
    counter = 0
    pagenum = 0
    more = True
    while(more):
        pagenum += 1
        print "PAGE "+str(pagenum)+": "+url
        req = urllib2.Request(url, None, headers)
        page = urllib2.urlopen(req).read()
        matches = regex_cache.findall(page)
        for match in matches:
            counter+=1
            tmp_req = urllib2.Request(match.replace('&amp;','&'), None, headers)
            tmp_page = urllib2.urlopen(tmp_req).read()
            print counter,": "+match
            f = open('files/'+str(counter)+'.html','w')
            f.write(tmp_page)
            f.close()
        #now check if there is more pages
        match = regex_next.search(page)
        if match == None:
            more = False
        else:
            url = "http://www.google.com"+match.group(1).replace('&amp;','&')
 
if __name__=="__main__":
    main()
 
# vim: ai ts=4 sts=4 et sw=4

Before using the script you need to adjust the search_term variable near the beginning of the script. In this variable goes the search term for which all the available cache would be downloaded. E.g. to retrieve the cache of all the pages of http://www.example.org you should set search_term to site:www.example.org

Save the following code under ~/.vim/syntax/google_gadgets.vim:

" Vim syntax file for Google gadgets module files.
" Maintainer: Guy Rutenberg <http://www.guyrutenberg.com>
" Last Change: 2008 Mar 28
"
 
:syntax clear
 
runtime! syntax/xml.vim
 
unlet b:current_syntax
:syntax include @HTML syntax/html.vim
:syntax region HTMLcode matchgroup=CDATA start=/<Content\s\+type="html"\s*>\_s*<!\[CDATA\[/ end=/\]\]>\_s*<\/Content>/ contains=@HTML
 
syn keyword javaScriptReserved _IG_Prefs getString getInt getBool getArray set setArray dump _IG_FetchContent _IG_FetchXmlContent _IG_FetchFeedAsJSON _IG_RegisterOnloadHandler _gel _gelstn _esc _unesc _trim _uc _min _max _hesc _args _toggle _IG_GetImage _IG_GetCachedUrl
let b:current_syntax = "google_gadgets"

And add the following line to your .vimrc

au BufRead <google_gadgets> runtime! syntax/google_gadgets.vim

The current syntax highlighting will highlight the XML part as XML, and if your gadget contains an HTML part, it will highlight it too (including CSS and JavaScript). I’ve also registered the core JavaScript library of the Google Gadgets API as JavaScript keywords so they will get highlighted too.

The line you added to the .vimrc file will automatically load the syntax highlighting for files of type google_gadgets. While this file type isn’t automatically detected, you can easily set one by typing :set filetype=google_gadget.

Tip: If you want some file to always be detected as google_gadget add the following modeline to it:

<!-- vim: set filetype=google_gadgets : -->

(you got to have modelines enabled for this to work).

Some things are still missing, such as highlighting for all the keywords and methods of the API, and automatic file type detection. I plan to address those issue as time will permit me. If you think I’ve left out some important functionality that should belong with this code, please comment or contact me.

Now we two possible ways to add the tracking code to each external link. The first one is to hack the MediaWiki internal parser for wiki code to generate additional code for each link. While this way will probably work the best it’s pretty complicated and not straightforward. The other way, which is the one will follow, is to use a small JavaScript snippet which will will iterate through the links in every page and add an “onClick” attribute to them with the tracking code.

<script>
var links = document.getElementsByTagName("a");
 
for (var i = 0; i < links.length; i++) {
        if (links[i].className=="external text") {
                addtrackcode(links[i]);
        }
}
 
function addtrackcode(obj) {
        obj.setAttribute('onClick',"javascript:urchinTracker('/outgoing/"+ obj.href.split("://")[1]+"');");
}
</script>

This code snippet should go between you Google Analytics code and the </body> tag, meaning you can add it to /wiki/skins/monobook.php if you haven’t changed the default skin for your MediaWiki. After the code is in place it may take up to 48 hours (usually you don’t have to wait at all) for the external links statistics to show up in the Google Analytics. In the Google Analytics the statistics for clicking on an external link which leads to http://www.google.com will show up as view of the page /outgoing/www.google.com. This script will also track download statistics of files which are linked from the wiki pages (like pdf’’s and such).

While this code does the job, it has a drawbacks. It only tracks the external links created by wiki code (e.g. [http://example.com]) as it uses CSS classes to determine what is external link. However this drawback isn’t very important and overall the script does a decent job tracking the external links in MediaWiki using Google Analytics.

<script src="http://www.google-analytics.com/urchin.js" type="text/javascript">
</script>
<script type="text/javascript">
_uacct="UA-xxxx-x";
urchinTracker();
</script>

in every page, preferably just above the </body> tag. The best way to do so will be to put the tracking code inside the base skin php file. That means that unless you changed the default skin for MediaWiki you need to edit /wiki/skins/MonoBook.php. In this file you will find the </body> tag towards the bottom of the file. Insert the tracking code just above it, save the file, and you’re done, as all pages will now show the script. Google Analytics will start gathering statistics usually in about 24-28 hours.

Update:
If you also want to track external links to files and other websites take a look at
Tracking Mediawiki External Links Statistics Using Google Analytics.