Bootstrapping Buildout Killing PYTHONPATH

written by Clayton Parker — on 10/25/10

— filed under: Buildout, Plone

We have been experimenting with using FreeBSD VMs for our development environment instead of relying on OS X. In the past week, we started experiencing really strange behavior with the buildouts on FreeBSD. The tables had turned so it seems, the builds on OS X are now working fine and the FreeBSD builds are failing. To add insult to injury, these issues were happening sporadically on other servers.

The main issue was that when the collective.recipe.plonesite part ran, anything that did a subprocess.call would fail giving an error like this:

Traceback (most recent call last):
  File "/usr/home/clayton/projects/my-buildout/bin/instance", line 200, in ?
    import plone.recipe.zope2instance.ctl
  File "/usr/home/clayton/projects/my-buildout/eggs/plone.recipe.zope2instance-3.6-py2.4.egg/plone/recipe/zope2instance/__init__.py", line 19, in ?
    import zc.recipe.egg
ImportError: No module named recipe.egg

I was finally able to reproduce the error along side a working copy of the same buildout. After having read about the PYTHONPATH changes in the latest version of buildout, I decided to diff the bin directory of the two instances. The diff of the buildout executable was particularly interesting.

--- old-buildout      2010-10-20 17:08:25.000000000 -0400
+++ new-buildout      2010-10-20 17:07:43.000000000 -0400
@@ -1,10 +1,18 @@
-#!/usr/local/bin/python2.4
+#!/usr/local/bin/python2.4 -S

 import sys
 sys.path[0:0] = [
-  '/usr/home/clayton/.buildout/eggs/setuptools-0.6c11-py2.4.egg',
-  '/usr/home/clayton/.buildout/eggs/zc.buildout-1.4.3-py2.4.egg',
-  ]
+  '/usr/home/clayton/sixfeetup/projects/sixfeetup/sfupsite-buildout/parts/buildout',
+  ]
+
+
+import os
+path = sys.path[0]
+if os.environ.get('PYTHONPATH'):
+    path = os.pathsep.join([path, os.environ['PYTHONPATH']])
+os.environ['BUILDOUT_ORIGINAL_PYTHONPATH'] = os.environ.get('PYTHONPATH', '')
+os.environ['PYTHONPATH'] = path
+import site # imports custom buildout-generated site.py

 import zc.buildout.buildout

You can see that the newly bootstrapped buildout has all the new PYTHONPATH setup in it. The workaround for this issue is to simply use the bootstrap.py -v option introduced in 1.3.0. Here is the command that allows our buildouts to run fine again:

$ cd path/to/buildout
$ python2.4 bootstrap.py -v 1.4.3

After doing that, the buildout gets the proper executable set up and runs fine.

Posted by Alex Clark on 10/25/10

Thanks for the post!

Yeah, this is annoying. I don't like the idea that Buildout is moving in the wrong direction, and I would love to see it fixed ASAP-ish.

Toward that end, I've gotten Zope core developer access and am hoping to help Gary Poster (the most active zc.buildout developer) with some fixes soon-ish.

Does Buildout have an issue tracker? If so, please open a ticket. If not, please add a ticket to the Plone trac under "installer" and assign it to me (technically not the right category but close enough).

Posted by clayton on 11/02/10

The tracker for buildout specific issues is on Launchpad:

https://launchpad.net/zc.buildout

Posted by Eric on 10/30/10

Hi. Thanks for writing about this. Has the bug been reported? Is there an issue/ticket I could track? Thanks.

Posted by clayton on 11/02/10

I'm still not 100% sure what the issue is, that's why I posted my findings here. I took a look through the tracker some more and found this issue:

https://bugs.launchpad.net/zc.buildout/+bug/492321

It seems to explain the issue that I was seeing. I added some notes to the ticket and bumped the "affects me" button. Hopefully it will get some attention soon.

Posted by mk on 01/07/11

Hi,

I guess it's why I can't buid the desired site. Please provice with more infos. Do I need to downgrade zc.buildout?

Thx,
MK

Posted by clayton on 01/10/11

If you are having the issue, you can do the following, where "python2.x" is the appropriate version of Python for your Plone build (2.4 for 3.x, 2.6 for 4.x+):

$ cd path/to/buildout

$ rm bin/buildout

$ python2.x bootstrap.py -v 1.4.3

$ bin/buildout

Now you should have the appropriate version of buildout installed.

Add comment

You can add a comment by filling out the form below. Plain text formatting.

Name