in Code, Geeklife, Linux, Python

Python: How to debug HTTP while using urllib2

import urllib
import urllib2

#this is just to prepare a dynamic uri (this is actual code from a system I'm building, sorry)
fileDownloadServiceURL = '%s://%s:%s/%s' % (transport,server,port,pathToController)
postData = {'URI':fileUri} #add more post stuff here
postData = urllib.urlencode(postData) #make sure you encode your post data

#add some custom headers if you need them
headers={"Host": server+':'+port,"Cookie":"JSESSIONID="+sessionId,"User-Agent":"Name of your User-Agent Here"}

#prepare your request, with headers and post data
req = urllib2.Request(fileDownloadServiceURL,postData,headers)

#and this is the magic. Create a HTTPHandler object and put its debug level to 1
httpHandler = urllib2.HTTPHandler()
#Instead of using urllib2.urlopen, create an opener, and pass the HTTPHandler
#and any other handlers... to it.
opener = urllib2.build_opener(httpHandler)

#User your opener to open the Request.
urlHandle =
#you'll end up with a file-like object... which I called urlHandle

The ouput will have useful debugging info about the HTTP connection.

connect: (localhost, 8080)
send: u'POST /arcturus-web/fileVariableDownload.service HTTP/1.1rnAccept-Encoding: identityrnContent-Length: 57rnConnection: closernUser-Agent: Temboo Twyla/Arcturus HTTP DownloaderrnHost: localhost:8080rnCookie: JSESSIONID=1b8xl8nozb2irnContent-Type: application/x-www-form-urlencodedrnrn'
send: ''
reply: 'HTTP/1.1 200 OKrn'
header: Content-Language: en-US
header: Content-Type: text/html; charset=ISO-8859-1
header: Connection: close
header: Server: Jetty(6.0.2)

Write a Comment


This site uses Akismet to reduce spam. Learn how your comment data is processed.