The realization of that had a profound impact in my life. As programmers, technically minded people that we are, we tend to focus on ourselves and rarely share our knowledge, except to show it off at conferences here and there. We do it mostly to please our ego. Dorneles was nothing like that. He didn’t present at many conferences, though he was always there as either a participant or an organizer. He didn’t write lengthy blog posts full of technical details. In fact, he didn’t blog at all since 2007. He didn’t create brilliant frameworks. Instead, his contributions were mostly invisible, through IRC, IM or Skype, directly between him and the person seeking his help. It was very common for him to carry about five parallel conversations over IM, helping people all over the world without breaking a sweat. He would happily pull out his laptop or phone at a table in a blink to teach someone a trick. The endless stream of messages on twitter following his death should serve as proof that his way was much more effective in causing an impact in the world than a horde of highly skilled programmers ever could.

As other people well put it, Dorneles could never lose that smile on his face. Nothing could make him curse or complain. He was a mixture of cheerfulness, innocence, tranquility and awkwardness that would immediately conquer people’s hearts with no effort. He was like that young little brother that makes you blush at a family dinner for being excessively goofy, but that you love so much you would never reprehend.

To me and my wife, Dorneles was closer than family. My wife suffered more from his loss than for the loss of her lovely grandma, or from her uncle that died a slow death from brain cancer a couple years ago.

In retrospect, I can now see that he did treat me like an older brother, always eager to take on my advice. When he was alone in a big empty office in Caxias do Sul, following the break with the other partners at X3ng I told him to move to Garibaldi because we could rent a similar office for much cheaper. I suggested that he should reunite with his wife and two kids, which were living with her mom, and that raising them in Garibaldi would be much better for their education and overall quality of life. I said it would be nice if he had a car so that he didn’t need to call a cab at 3am when his kids were sick (he refused to buy a car for years, since he had cars stolen several times). At a time he was having trouble getting regular pay, I suggested he should work for Enfold. I told him to go to more conferences and sprints, so that he could put faces to the names on #plone. He was easily convinced. And all of these things had a great impact in his life.

I regret not spending as much time with him as I could in the last year. But many are the good memories that we’ve shared. Back in November 2009, when my backpack with laptop and cameras containing all the pictures from my vacation in New York got stolen on our first day in Houston he calmly told me: “Don’t worry, you can buy all those things again. They didn’t take from you the most precious thing: your life.”

I regret that he didn’t find the time to visit my 3mo old twins. But I’m glad that after my visit last Saturday, he went to visit his 2 weeks old nephew and his dad on the Sunday. I’m glad that we spent a good 2h last Saturday sitting around, chatting about the most diverse subjects while I was at his place waiting for the rain to stop.

Dorneles was a sushi lover. He once walked for tens of blocks in Houston on a cracking hot summer day just to eat sushi, and lived to tell the tale. That’s perhaps the most un-American thing he could do. The first thing we did when we met in Houston in November 2009 was to go out for sushi. When Nate came to Brazil, he took him out for sushi, and that’s perhaps the most un-Brazilian thing he could do. His love for sushi passed on to his kids. For several weeks I told my wife that we should invite his family out for sushi. We were waiting for the right occasion, and it never happened. In his memory, I think we should do it anyway.

This is an attempt at summarizing all the activity that happened in the last five months, though it’s far from being a short summary. If I had to pick a two words to describe my first five months at Canonical, it would be “Pure Awesomeness“. For a more detailed view, grab a cup of joe or your favorite other beverage and keep on reading.

Today is a special day. Exactly five months ago, on January 5th, I joined Canonical to work on the Landscape project.

It has been quite a ride so far, with two sprints with the Landscape team, AllHands and UDS in Barcelona just a week ago, and lots of excitement about the future. Saying that I’m completely stunned by the work everyone at Canonical has been putting together and how much the teams have grown in the last few months doesn’t do enough justice to it.

At AllHands and UDS we got a short preview of the things that are coming out in the next cycle and beyond. An example of that is the newly formed Design and User Experience (DUX) team which will not only be focusing on Ubuntu itself, but on many other areas across Canonical and the whole Open Source community in general.

At UDS, the DUX team had a special ‘booth’ where any person from any project could walk to them and get advice about their personal or favorite application. One person which has favorably used such advice already was Seif Lofty, from Gnome Zeitgeist. I met Seif during breakfast at the first day of UDS and I cannot describe how excited he was about simply *being* at UDS. And he was certainly twice as happy when he left.

Another person I had the joy to meet was David Siegel, of Gnome Do fame. We teamed during AllHands on the infamous “Fun In The Woods” activity, which had some people walking like zombies on the day after.

Even more importantly, I was able to meet many many more other colleagues from different teams and wrap up some loose ends from tasks that I got started during the first couple months. I’m definitely impressed by the amount of plain brilliant people that are part of Canonical as of today.

In a sense, being at a Canonical event is very much like being at a Plone conference. Everyone seems to be very receptive about new ideas and very friendly and laid back. And as a bonus, I was able to exercise my (not so secret now) power of throwing some crazy ideas around and see how they influence people. And man, I’m already impressed by the outcomes, just a week after the fact.

To me, this has been the most rewarding thing so far, to be able make big contributions not in lines of code, but in ideas that can make a concrete difference in the hands of the right people. This is something that can only be possible at a company the size Canonical is at the moment, where it’s just big enough that you can grab a mind or two to push an agenda without affecting the rest of the team and still small enough that you can influence decisions.

As my colleague Jamu would best describe, “I’m PUMPED!”. :)

But that’s not all. Software-wise, I was able to make some big contributions too. The Landscape team just finished a 6 month development cycle that brought many cool features to life. I’m really happy with that, and specially with the speed that this team can get features from the black board into reality. It’s also a much different environment than what I was used to, with very well-defined and refined processes for ensuring the overall quality of anything that is produced. One process that I’m specially enjoying is the requirement for two positive reviews before landing a branch. I hope to talk more about that soon (that is, sooner than 5 months from now *wink*).

As for my role in the team, it is quite different than what I’m used to. I’ve been focusing a lot on the UI aspects of Landscape, on ways to make things more obvious and more streamlined. I’ve been also writing a ton of Javascript, and collaborating with other teams to define better policies for Javascript testing in general. And finally, we will now have a person from the DUX team dedicated to working with us, which will push work on the Landscape UI even further.

I also had the chance of interacting with the Launchpad team, which has a much more refined process due to the size of their team. Over at Launchpad, I started a branch back in mid-December, even before starting at Canonical, to allow Launchpad to use the Chameleon Template Engine.

That was another wild ride, and during the course of this project I was able to contribute tons of fixes upstream to Malthe Borch to make Chameleon even more compatible with plain old ZPT. In fact, it is so compatible at the moment that due to the magic of z3c.ptcompat Launchpad will be able to run *both* Chameleon and ZPT with the flip of an environment variable. Even more stunning, the changes required to code were minimal, basically changing imports to use z3c.ptcompat, and in templates we’ve had to fix some non-XHTML compliant ones and remove unused i18n tags. I am happy to announce that this branch will soon be merged (it was submitted to PQM, successfully accepted and is waiting to land the buildbot queue). The bad news is that not all tests pass at the moment with Chameleon enabled, but we will be dogfooding and fixing those tests as we go. It was too much pain already to maintain a nearly 6 month old branch outside the main tree. ;)

I am really interested in many of the things that the Launchpad team is doing, process-wise. The PQM seems like a very nice idea for a bigger team like theirs, though it would probably be useful to our smaller team in Landscape too, and to others in general. Hopefully I will get a chance to explore it more and talk about it during the upcoming FISL 10, in Porto Alegre.

Lastly, but not least important, I’m also working on getting nightly builds of the Bzr Installer for Windows rolling, and a more streamlined process for the official builds. Karl Fogel, of Producing OSS fame, and our Launchpad Ombudsman is making sure I keep my promises about that, which is yet another great incentive.

All in all, there’s of course a ton of things I forgot to talk about and which happened in the last 5 months, but this post is already getting too long so I will stop right here and save some of the meat for a future one. Stay tuned!

What’s more important to end-users though is that third-party libraries are usable with Python 3.0, and not many of them are right now. One particular library has a special place in my heart, and is getting pretty close to being fully compatible with Python 3.0: PyWin32, the Python for Windows Extensions.

I first asked Mark about a year ago if he ever thought that PyWin32 would be ported to Python 3.0, and he was very pessimistic about it. Things changed recently though, and with the “2to3″ library getting more mature, so did the chances of PyWin32 being ported to Python 3.0 increase.

Mark sent a status update with a detailed explanation of the process mid-November (http://tinyurl.com/py3k-win32-status-1) and another two (http://tinyurl.com/py3k-win32-status-2, http://tinyurl.com/py3k-win32-status-3) more recently. I recommend reading those to anyone thinking about porting their libraries to Python 3.0, there’s a lot of great insight there.

To summarize, the great news is that PyWin32 will soon be capable of running on Python 3.0 without being forked: a single installer will work for Python 2.x and Python 3.x thanks to the “2to3″ translation library!

So many thanks to Mark Hammond, Roger Upole and Vernon Cole for their work. To me, the fact that a gigantic and complex library like PyWin32 can be made to run on Python 3.0 at all is the best validation that the Python Developers could possibly get for their work. You did a really amazing job!

Oh, and in case you didn’t notice, PyWin32 is now hosting a Bazaar mirror on Launchpad (https://code.edge.launchpad.net/pywin32), and that’s where Mark is hosting his py3k-integration branch too. Go Launchpad!

This week, with the Plone Conference 2008 going on in Washington and the fact I did not attend this conference, I finally made enough time available to get my hands back into Zope again.

So it’s my pleasure to report that after about a week of hard work and a handful of workarounds that will need future cleaning, Zope 2 does now startup under Python 2.6.

My expectation with this now out of my way is that other developers will start looking at adjusting third-party applications (hint: Plone) to run on Python 2.6 as well.

The code is available on the gsoc-python-2.5 branch of the Zope repository, and the lucky guys at the Plone Conference will be amongst the first to hear the good news through Matthew Wilkes, which registered for a stealth Lightning Talk which should get at least some people puzzled. I would love to be there and see their faces when Matthew unveils the surprise.

Big thanks go to Tres Seaver for helping me figure out a change needed in Zope’s private version of ‘medusa’ and to Ranjith Kannikara, our GSoC student through the Zope Foundation, and his friends which did a good chunk of the hard work, namely figuring out changes to the C-based ‘Acquisition’ module and cleaning up string exceptions.

NOTE: This is not intended for production yet, it is just a technology preview to get developers to try out and start porting their own projects. Many bugs still exist, some of which might take up to a month to get fixed. Most importantly, the RestrictedPython implementation has not been fully audited for the new builtins and language constructs introduced since Python 2.4.

EnSimpleStaging is based on ZopeVersionControl, another great product which provides for keeping a repository of versioned objects inside the ZODB. One issue we’ve hit constantly with EnSimpleStaging was that certain methods from ZopeVersionControl depend on an actual transaction being commited to properly detect that an object has changed. While in real life this specific method will always work flawlessly due to the way using ZopeVersionControl and EnSimpleStaging works, in a test case it was very hard to test this, due to the fact that a test case is run within a single transaction which is aborted at the end of the test.

Sometime ago I’ve learnt that ZODB has a nifty feature: it is possible to wrap any Storage instance with a DemoStorage. After you do that, you can commit changes and they will be committed to the DemoStorage but not to the base Storage. That would solve not only my problem with ZopeVersionControl, but can potential solve a whole lot set of problems due to expensive setup methods in test cases.

That fact has lived in the back of my mind for longer than two years, until today I set some time aside to investigate a way to implement this cleanly in a way that could be used for my tests. Searching around, I’ve found at least a couple occurrences of other people trying to do the same. I didn’t find any solution though, so that either indicates they just gave up or that they found a solution but didn’t care enough to share it. In fact, the title of this post was borrowed from one of the message subjects, in hope that it will make it easier for other people to find a solution in the future.

My solution (with a test do demonstrate that it works) was based on Jim Fulton’s suggestion to look at zc.selenium. I’ve simply adapted it to the Zope 2 reality, using ZopeTestCase’s provided hooks and `Zope2.bobo_application` (which is the Zope 2 equivalent of the publication object from Zope 3).

Hopefully this will be of use to more people, specially to the folks that were seeking for ideas about how to implement this. It would be even better to just add this to ZopeTestCase proper, I think. Maybe someone will be interested in doing that.

me: i mean, i had made a list of things already

Alexander: yup, but I’m sure there are more

me: those are the high-level yeah
for example, you can run with the python security policy
implementation, so that’s one thing less
acquisition needs to be ported though, or at least faked :)

Alexander: what about the __parent stuff that just landed in 2.11 (or
was it trunk?)

me: that’s implemented in Acquisition yeah
ExtensionClass, Acquisition and persistence are the 3 top modules
oh, BTree is a nasty one
probably better off trying to go to a non-ZODB based Zope by that time

Alexander: yeah, I suspect we should ditch the ZODB if we want to be
on App Engine

me: at the same time, there’s this project about porting Zope to Jython
if that’s successfull, it should run on pure python too
or maybe not, depends on which way this student goes

Alexander: right, the App Engine concerns are more than just what runs
as pure Python, though :)
database, memory usage, etc

me: but, what about the relstorage backend?
idea

Alexander: that’s why I included Martijn ;)

me: ok
so, yeah, those are my ideas
can’t think of much else

Alexander: ok
that’s a great start, thanks
now make sure the email or blog post covers the additional thoughts ;)

me: maybe the DirectoryViews stuff
the filesystem access is restricted right?
but i guess you can read files just fine?

Alexander: I have no idea :)

me: oh, k
number of files would be a problem
we should have all products running off zipped eggs by now
that would reduce the amount of files
but unfortunately no one worked on that

Alexander: I have to run, but feel free to continue in email ;)

me: ok

Alexander: trunk of Plone is mostly eggs, no?

me: but nothing is ready to load files off zipped eggs

Alexander: I see

me: i believe mcdonough did some work on that

Alexander: ok, see you later :)

me: ok
zcml would need to grow support for zipped eggs too

If you have more ideas, feel free to comment.

I am amazed to see the progress he and the folks helping him had so far. This is definitely all that I dreamed for when I started with the idea in January 2006. It is great to see that even though ideas take a little longer to get implemented in the Plone community that they do not simply get abandoned, but there’s always people willing to pick up an idea and turn it into reality.

Many thanks for Tarek for all of his work. Once this gets rolled out into the new plone.org it will make the life of us developers so much easier when making releases that we will ask ourselves “how could I live without this for so long”.