Translate

Archive for the 'Python' Category

PyQt4: Using QMutex vs QMutexLocker.

Sunday, November 25th, 2007

Here’s some code for my future reference on how to use QMutex or QMutexLocker.

Lessons Learned:
* Use QMutex to protect data, not code. Try not to lock hughe amounts of code within a function with mutex.lock(), mutex.unlock(), if for any reason you forget to release the lock you’ll be in trouble. Use the mutex directly only when you know what it is that you want to protect concurrent access from.
* When you have a complex function and you don’t want to worry about what to protect and when to release the lock (on exceptions thrown, before returns,etc), you can create an instance of QMutexLocker and it should release the mutex lock upon destruction… this takes us to the next lesson
* When using a QMutexLocker, DO NOT make the QMutexLocker an attribute of your class, otherwise, the reference will live after the method finishes and the lock won’t be released.

Here’s some code.

from PyQt4.Qt import QObject, QMutex, QApplication, QThread, QMutexLocker
import sys

class MutexTestSubject(QObject):
	'''
	Class that uses a QMutex to synchronize
        access to its add(),substract() methods.

        This works perfectly fine.
	'''
	def __init__(self):
		QObject.__init__(self)
		self.MAX_LIMIT = 100
		self.MIN_LIMIT = 0
		self.counter = 50
		self.mutex = QMutex()

	def add(self):
		self.mutex.lock()
		if self.counter < self.MAX_LIMIT:
			self.counter = self.counter + 1
		self.mutex.unlock()

	def substract(self):
		self.mutex.lock()
		if self.counter > self.MIN_LIMIT:
			self.counter = self.counter - 1
		self.mutex.unlock()

	def printStatus(self,thread):
		print "Counter:",self.counter," - Thread:",id(thread)

		if self.counter > self.MAX_LIMIT+1 or self.counter < self.MIN_LIMIT:
			print "Stopping Threads, Max Surpassed, Not Thread Safe. Last Thread:",id(thread)
			sys.exit()

class MutexLockerTestSubject(QObject):
	'''
	Class that attemps to synchronize thread
 	access to its add(),substract() methods with
	the QMutexLocker object.
	'''
	def __init__(self):
		QObject.__init__(self)
		self.MAX_LIMIT = 100
		self.MIN_LIMIT = 0
		self.counter = 50
		self.mutex = QMutex()

	def add(self):
		#VIP: DO NOT MAKE mutexLocker an attribute of your class.
		#other wise it won't be destroyed and the lock will never be released.
		mutexLocker = QMutexLocker(self.mutex)
		if self.counter < self.MAX_LIMIT:
			self.counter = self.counter + 1

	def substract(self):
		mutexLocker = QMutexLocker(self.mutex)
		if self.counter > self.MIN_LIMIT:
			self.counter = self.counter - 1

	def printStatus(self,thread):
		print "Counter:",self.counter," - Thread:",id(thread)

		if self.counter > self.MAX_LIMIT+1 or self.counter < self.MIN_LIMIT:
			print "Stopping Threads, Max Surpassed, Not Thread Safe. Last Thread:",id(thread)
			sys.exit()

class AdderWorker(QThread):
	def __init__(self, mutexTestObject):
		self.mutexTestObject = mutexTestObject
		QThread.__init__(self)

	def run(self):
		while(True):
			self.mutexTestObject.add()
			self.mutexTestObject.printStatus(self)

class SubstractorWorker(QThread):
	def __init__(self, mutexTestObject):
		self.mutexTestObject = mutexTestObject
		QThread.__init__(self,mutexTestObject)

	def run(self):
		while(True):
			self.mutexTestObject.substract()
			self.mutexTestObject.printStatus(self)

if __name__ == "__main__":
	USE_MUTEX_LOCKER = True #switch this to use regular mutexes vd QMutexLocker
	app = QApplication(sys.argv)
	mutexTestObject = MutexTestSubject() if not USE_MUTEX_LOCKER else MutexLockerTestSubject()

	adderThread  = AdderWorker(mutexTestObject)
	substracterThread = SubstractorWorker(mutexTestObject)

	adderThread.start()
	substracterThread.start()

	sys.exit(app.exec_())
Digg This
Reddit This
Stumble Now!
Buzz This
Vote on DZone
Share on Facebook
Bookmark this on Delicious
Kick It on DotNetKicks.com
Shout it
Share on LinkedIn
Bookmark this on Technorati
Post on Twitter
Google Buzz (aka. Google Reader)

Python Reference: Binary Operators

Saturday, July 28th, 2007

Python binary operators are pretty much the same as in any other language, however I notice most programmers tend to waste a lot of memory by creating lots and lots of properties say in DB tables, or Objects and using the wrong datatypes. I think its elegant to use the concept of binary flags, for example, if you have an object that has around 8 or 16 boolean properties, you can store their state in 1 byte or 2 bytes (1 char or 2 char fields), and turn on/off the bits on those fields.

Usually, if you have a binary field, and you want to, turn on bits, toggle bits, and check bits, you do the following.

Suppose “config” is an 8 bit number, and you want to modify this bits individually, say config=4 (0b100) and you want to turn on the rightmost bit to have 0b101 (which is a 5) you could do the following

>>> #Uses the | operator
... def turnBitOn(config, binaryFlag):
...   return config | binaryFlag
...
>>> config = 4
>>> config = turnBitOn(config,1)
>>> print config
5

If you want to check if a Bit is turned on, just use the “&” operator, if the result is the same as the bit you’re comparing, then its turned on.

>>> def checkBit(config,binaryFlag):
...   return binaryFlag == config & binaryFlag
...
>>> print checkBit(config,1)
True
>>> print checkBit(config,2) #checks 0b010 in 0b101
False

What if you just want to toggle a bit, no matter what’s in there?
Just use binary XOR, the “^” operator.

>>> def toggleBit(config,binaryFlag):
...   return config ^ binaryFlag
...
>>> print config
5
>>> config = toggleBit(config,1)
>>> print config
4
>>> config = toggleBit(config,1)
>>> print config
5

And now, the last basic operation would be to turn off a bit. For this, you should do a NAND operation. The Binary Not in python as in most programming languages is the “~” operator. This is how you can use it to turn off bits.

You have to do it in conjunction with the & operator, sort of doing a NAND

>>> 5 & (~1)
4
>>> 5 &~ 1
4

So we could define our turnOffBit function as follows:

>>> def turnOffBit(config,binaryFlag):
...   return config & (~binaryFlag)
...
>>> print config
5
>>> config = turnOffBit(config,1)
>>> print config
4
>>> config = turnOffBit(config,1)
>>> print config
4

Hope this makes a good reference for those trying to make the most out of their bytes.

Digg This
Reddit This
Stumble Now!
Buzz This
Vote on DZone
Share on Facebook
Bookmark this on Delicious
Kick It on DotNetKicks.com
Shout it
Share on LinkedIn
Bookmark this on Technorati
Post on Twitter
Google Buzz (aka. Google Reader)

Python: How to debug HTTP while using urllib2

Friday, July 27th, 2007
...
import urllib
import urllib2

#this is just to prepare a dynamic uri (this is actual code from a system I'm building, sorry)
fileDownloadServiceURL = '%s://%s:%s/%s' % (transport,server,port,pathToController)
postData = {'URI':fileUri} #add more post stuff here
postData = urllib.urlencode(postData) #make sure you encode your post data

#add some custom headers if you need them
headers={"Host": server+':'+port,"Cookie":"JSESSIONID="+sessionId,"User-Agent":"Name of your User-Agent Here"}

#prepare your request, with headers and post data
req = urllib2.Request(fileDownloadServiceURL,postData,headers)

#and this is the magic. Create a HTTPHandler object and put its debug level to 1
httpHandler = urllib2.HTTPHandler()
httpHandler.set_http_debuglevel(1)

#Instead of using urllib2.urlopen, create an opener, and pass the HTTPHandler
#and any other handlers... to it.
opener = urllib2.build_opener(httpHandler)

#User your opener to open the Request.
urlHandle = opener.open(req)

#you'll end up with a file-like object... which I called urlHandle
...

The ouput will have useful debugging info about the HTTP connection.

connect: (localhost, 8080)
send: u'POST /arcturus-web/fileVariableDownload.service HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 57\r\nConnection: close\r\nUser-Agent: Temboo Twyla/Arcturus HTTP Downloader\r\nHost: localhost:8080\r\nCookie: JSESSIONID=1b8xl8nozb2i\r\nContent-Type: application/x-www-form-urlencoded\r\n\r\n'
send: 'URI=temboo%3A%2F%2Fwww.one.com%2Ffiles.f%2Fmapping1.m.var'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Language: en-US
header: Content-Type: text/html; charset=ISO-8859-1
header: Connection: close
header: Server: Jetty(6.0.2)
Digg This
Reddit This
Stumble Now!
Buzz This
Vote on DZone
Share on Facebook
Bookmark this on Delicious
Kick It on DotNetKicks.com
Shout it
Share on LinkedIn
Bookmark this on Technorati
Post on Twitter
Google Buzz (aka. Google Reader)

Note about Signals and PyQt4.

Monday, July 9th, 2007

I keep making the mistake of sending PyQt_PyObjects instead of sending actual Qt4 objects on signals that are defined by Qt like that.

Bottom line:

If a signal has been defined by Qt, to send Qt objects, just copy and paste it, do not try to override it by exchanging the Qt objects for PyQt_PyObject.

This is because PyQt_PyObject is used to represent regular Python objects, therefore, original Qt4 classes will never emit signals under those weird signatures, Qt4 seems to be very strict on how it matches signals.

So when you see something like this on the documentation:

Qt Signals
void currentItemChanged (QTreeWidgetItem *,QTreeWidgetItem *)

Define your signal for example like this, and it will work:

SIGNAL_ITEM_CHANGED = SIGNAL('currentItemChanged (QTreeWidgetItem *,QTreeWidgetItem *)')

Do not do this, cause it won’t work:

SIGNAL_ITEM_CHANGED = SIGNAL('currentItemChanged (PyQt_PyObject, PyQt_PyObject)')

(Keep in mind PyQt_PyObject is used to represent Python objects only, and its useful only when YOU define and emit your own signals)

then connect it from a QTreeWidget item (or derived object) to a listener object’s method like this:

QObject.connect(myQTree,SIGNAL_ITEM_CHANGED,myListener.onItemChanged)

Your object’s listener method should look like this:

def onItemChanged(self, current, previous):
  #do what you gotta do, current and previous will be QTreeWidgetItems
Digg This
Reddit This
Stumble Now!
Buzz This
Vote on DZone
Share on Facebook
Bookmark this on Delicious
Kick It on DotNetKicks.com
Shout it
Share on LinkedIn
Bookmark this on Technorati
Post on Twitter
Google Buzz (aka. Google Reader)

How to update file timestamps in Python

Tuesday, May 29th, 2007

Sometimes you can be real picky like me about timestamps of files, for example, during my wedding we had a few digital cameras, and one of the cameras had its internal clock 4 hours behind. So what better way for a lazy guy like you to change timestamps than writing a short python script to fix the problem.

I wrote this script on the same folder where the pictures with the lagged modification times existed:

import os
import time
from stat import *

#returns a list of all the files on the current directory
files = os.listdir('.')

for f in files:
  #my folder has some jpegs and raw images
  if f.lower().endswith('jpg') or f.lower().endswith('crw'):
    st = os.stat(f)
    atime = st[ST_ATIME] #access time
    mtime = st[ST_MTIME] #modification time

    new_mtime = mtime + (4*3600) #new modification time

    #modify the file timestamp
    os.utime(f,(atime,new_mtime))

Very nice, I like

Digg This
Reddit This
Stumble Now!
Buzz This
Vote on DZone
Share on Facebook
Bookmark this on Delicious
Kick It on DotNetKicks.com
Shout it
Share on LinkedIn
Bookmark this on Technorati
Post on Twitter
Google Buzz (aka. Google Reader)

Recordando algoritmos, Quicksort

Thursday, May 17th, 2007

Leyendo no se que en internet, lei mencionar algo sobre preguntas de entrevistas de trabajo, y mencionaron que siempre es bueno saber como funciona Quicksort. Por algun motivo, el algoritmo de ordenamiento que siempre recuerdo es Bubble sort, pero todos sabemos que no es el mejor. So decidi repasar y leer como funciona quicksort.

Para los que no recuerdan quicksort es simplemente hacer 3 listas con la lista principal [PRINCIPAL]. Estas listas se generan bajo un criterio bien sencillo. Elige un numero cualquiera del arreglo, la primera lista tendra todos los numeros menores a ese numero [MENORES]. La segunda lista tendra todos los numeros que sean de igual valor a ese numero [IGUALES], y la tercera los mayores [MAYORES].

Una vez que tienes estas 3 listas, el resultado de Quicksort es el siquiente (Donde ++ es concatenar una lista):

Quicksort([MENORES]) ++ [IGUALES] ++ Quicksort([MAYORES])

Como veras, el algoritmo es recursivo, y llegara el momento que no encuentres numeros menores, iguales o mayores, asi que cuando Quicksort recibe una lista de un solo elemento, devuelve ese elemento.

En python esto sale en 4 lineas con list comprehension, literalmente:

def qs(lista):
   if len(lista) <= 1: return lista
   p = lista[0] #el pivote
   return qs([x for x in lista if x < p]) + [x for x in lista if x == p] + qs([x for x in lista if x > p])

Pudieras aplicar este algoritmo a cualquier tipo de objeto en Python si para ese objeto defines las funciones de comparacion.
Demasiado arrecho, viva python.

PS: Despues de escribir esto, encontre una implementacion aun mas corta, en la que no utilizan list comprehension, utilizan
funciones lambda, en vez de hacer [x for x in lista if x < pivote] hacen filter(lambda x,y=pivote: x < y,lista), eso devuelve los elementos que cumplen con la funcion lambda en la lista, es decir filter(funcion, lista)

Digg This
Reddit This
Stumble Now!
Buzz This
Vote on DZone
Share on Facebook
Bookmark this on Delicious
Kick It on DotNetKicks.com
Shout it
Share on LinkedIn
Bookmark this on Technorati
Post on Twitter
Google Buzz (aka. Google Reader)