Tuesday, January 19, 2010

Big Scary Software - python runtime, filesystems, etc

Mort (now at Nottingham) pointed us at some educational videos in the last couple of days

1.The Giant Interpreter Loop that python runtime uses is not really something that admits of concurrency, but also has a lot of weirdnes about it...
(sorry - as per comment - GIL = Global Interpreter Lock, not Giant Incense Lamp)

2. Making sure things really are written to stable storage or disk, as we used to call it, is really not quite as simple (or obvious) as you might think

3. not even on windows

I guess these should be like public health warnings, a bit like the German and French governments recent IE warnings

3 comments:

Peter said...

GIL = http://en.wikipedia.org/wiki/Global_Interpreter_Lock

Sachin said...

The Python Multi-Process module is a possible work-around in order to avoid the GIL issues and also utilize multiple CPUs.

Offcourse you'd have to pay the additional process overhead, but the module's syntax and interface is very similar to the Threading interface so switching to it is not a big issue. In fact it is called the Process-based "threading" module in Python documentation. You also get some of the advantages of processes, like not having to deal with shared memory for communication (as is the case in threads) if you dont want to.

More here
http://docs.python.org/library/multiprocessing.html

jon crowcroft said...

thanks for GIL correct expansion and further pointers...

of course, its not really fundamental to python as a language ...its sometimes sociologically fun to see people fighting over programming langauges (language features, implementations etc!)...