Python: How to debug HTTP while using urllib2
...
import urllib
import urllib2
#this is just to prepare a dynamic uri (this is actual code from a system I'm building, sorry)
fileDownloadServiceURL = '%s://%s:%s/%s' % (transport,server,port,pathToController)
postData = {'URI':fileUri} #add more post stuff here
postData = urllib.urlencode(postData) #make sure you encode your post data
#add some custom headers if you need them
headers={"Host": server+':'+port,"Cookie":"JSESSIONID="+sessionId,"User-Agent":"Name of your User-Agent Here"}
#prepare your request, with headers and post data
req = urllib2.Request(fileDownloadServiceURL,postData,headers)
#and this is the magic. Create a HTTPHandler object and put its debug level to 1
httpHandler = urllib2.HTTPHandler()
httpHandler.set_http_debuglevel(1)
#Instead of using urllib2.urlopen, create an opener, and pass the HTTPHandler
#and any other handlers... to it.
opener = urllib2.build_opener(httpHandler)
#User your opener to open the Request.
urlHandle = opener.open(req)
#you'll end up with a file-like object... which I called urlHandle
...
The ouput will have useful debugging info about the HTTP connection.
connect: (localhost, 8080) send: u'POST /arcturus-web/fileVariableDownload.service HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 57\r\nConnection: close\r\nUser-Agent: Temboo Twyla/Arcturus HTTP Downloader\r\nHost: localhost:8080\r\nCookie: JSESSIONID=1b8xl8nozb2i\r\nContent-Type: application/x-www-form-urlencoded\r\n\r\n' send: 'URI=temboo%3A%2F%2Fwww.one.com%2Ffiles.f%2Fmapping1.m.var' reply: 'HTTP/1.1 200 OK\r\n' header: Content-Language: en-US header: Content-Type: text/html; charset=ISO-8859-1 header: Connection: close header: Server: Jetty(6.0.2)
