PyConBrasil 3 is over. It was a huge success, and I hope to post some comments about it. But for now, life is back and she’s a bitch.

 I had an issue with an Apache output filter written in Python, using mod_python, when together with Apache. After several days (a whole week to be more precise) trying to figure out what was causing it I seem to be able to find the problem, apparently. Here’s my post to the mod_python mailing list detailing the issue.

Ok, I was finally able to figure out part of the problem and a solution for it. The issue is that if the response fits in ‘io_buffer_size’ which defaults to 8k, then the output filters get all called in the same iteration.

In particular, there’s one filter called ‘HTTP_HEADER’ which is responsible for emitting the headers. Obviously, to make things work right, it should be the last one to be called, which it is apparently. However, when the response is larger than io_buffer_size, the filters get called multiple times.

 Unfortunately, HTTP_HEADER is one such filter, so the first time around the mod_python filter gets called, but hasn’t seen EOS, so it doesn’t change the headers. Then the HTTP_HEADER gets called and sends the headers. Then after that you can’t change the headers anymore.

My initial hack was to set ‘req.assbackwards’, which prevented the HTTP_HEADER from doing it’s thing, and then when the mod_python filter sees EOS then I set ‘req.assbackwards’ to 1 and call req.add_output_filter(‘HTTP_HEADER’). That works, but depends on filter implementation internals, so I felt I needed a better fix.

Then I stumble upon ‘mod_filter’, which has lots of interesting configuration knobs. The interesting one is ‘FilterProtocol’, which lets you specify if your filter will change the content length and if it supports byte ranges, and also that you want your filter to be the first one in the chain. So I toggled those settings, and it magically works.

Here’s my configuration file: https://svn.enfoldsystems.com/browse/public/enfold.lxml/trunk/docs/sample-apache.conf?rev=1321&view=auto

It would be great to document this somewhere, so that the next person doesn’t have to suffer that much. Maybe add some notes to mod_python docs? I will certainly write a blog entry.

Hopefully that will save someone’s time when trying to write output filters for Apache with mod_python. I just hope more people would post their solutions when they have to work around or fix complex issues. It would make everyone’s life much better.