1. Would surely be very useful.
    Smells like the reason I like dselect in Debian so.

    However, for the next year or so people will most likely be uncertain of it. They might think: I better get this right, so I'll download it manually.

    Screenscrapping? What's wrong with XML-RPC?
      posted by Peter Bengtsson at 02:09:32 PM on January 25, 2004  
  2. Sounds like CPAN, which is definitely not a bad thing. That said, one problem I have with auto-install systems is that they're installing a bunch of code on my system that could potentially be malicious. Since anyone can upload code to PyPI, how can I be sure the code I am installing doesn't have malicious intent? At the very least I would want some kind of cryptographic signing so I can be sure that the code downloaded by the installer is the code that was released by the package maintainer.
      posted by Simon Willison at 02:38:25 PM on January 25, 2004  
  3. We can pretty much do this on OS X, but the only two module databases (not connected to PyPI) are maintained by Jack Jansen and I.

    When I talked about adding crypto to the system people kept bringing up export regulations, so we didn't do that (md5 on the packages, no signing of the package list itself though).
      posted by Bob Ippolito at 02:45:27 PM on January 25, 2004  
  4. I thought the crypto restrictions were pretty simple now? Certainly md5 is okay, and I think AES is as well.

    I'm not sure exactly what signing would mean. Ultimately it could mean something like the Debian keyring, where we create a trusted network of Python programmers, but I don't know if we could really maintain or justify that level of security for a looser community like Python programmers.

    On the matter of Debian, it would be nice to be able to fetch a distutilified package, create a debian package from it, and then install it, all in one command. Though I don't think Debian has all the tools to automatically package distutil packages yet. Though I suppose everything is in place for doing that with RPM.

    Of course, if we do that for Debian packages, next we want a way for PyPI to generate a packages list, and allow apt-get upgrades.

    As we go further, we start to require more metadata, like dependencies. E.g., I recently installed mechanize, but went back and forth as I got all the dependencies (of which there were quite a few). I would expect the dependency situation to get worse as this style of software distribution grows, as people will start to create software that is more tied into a larger network of software.
      posted by Ian Bicking at 05:11:12 PM on January 25, 2004  
  5. Yes, one of the few useful things the Clinton adminstration did was (towards the end) fix the crypto regulations. Years later, even MIT has caught up, and distributes kerberos now.

    The fun bit is that it is much *easier* for open source than for proprietary stuff.

    And yeah, pypi-get is a good start, but I want dh_make_python (like dh_make_perl, which takes a module name, downloads from CPAN, and builds a package structure around it - you still want to review it, but it does all of the "stupid stuff", ie the easily automatable parts of translating one metadata space (CPAN's) to another (Debian's)...)
      posted by Mark Eichin at 01:30:55 PM on January 26, 2004  
  6. I'd say that additional fields would be required in the database to support this (this is not a big deal, it just needs to be discussed and specified). Also, the gpg web of trust is slowly expanding in the Python community, so trusted gpg signing could be an option (ie. having a pypi or python.org key which is signed by Barry Warsaw, and so on).

    Note that the catalog-sig and distutils lists have discussed stuff like this endlessly :)

      posted by Richard Jones at 02:38:38 PM on January 26, 2004  
  7. """Note that the catalog-sig and distutils lists have discussed stuff like this endlessly :) """

    But apparently that didn't result in any working implementation. Hmm, I better search the archives first, maybe there is a good reason *why* something like this has not been written yet.
      posted by Hans at 03:13:17 PM on January 26, 2004  
  8. Very good idea!! It would be great!! The only thing I could think of that would be even better would be to include dependency-information and automate the process even further.
      posted by Thomas Weholt at 03:03:17 AM on January 27, 2004  
  9. I have posted this idea in the lang.python forum, and got no real answer. I think it have not be done by laziness.
    MacPython have it, ActivePython had it (and worked pretty fine), but the OpenSource distro have not it yet.
    The MacPython and ActivePython systems are a proof that there is no technical problem to do it. It was not done yet because the pressure to do it was not enough.

    Personally I think it is VERY important, specially to help with python modules dependencies allowing semi-auto installs if required modules are not aviable.

    I would be glad to help on such a thing (even if my agenda is somewhat busy now).

    RodrigoB.
      posted by rodrigob at 03:57:19 AM on January 27, 2004  
  10. MacPython *is* the "OpenSource" distro. There's no more or less than what's in the python.org CVS tree.
      posted by Bob Ippolito at 06:23:33 AM on January 27, 2004  
  11. The main reason it hasn't been done is the same reason that an index hadn't been done for so long. It's not for want of good ideas :)

    People simply have better things to do with their time. I only implemented PyPI because I found myself with some small amount of spare time, and was prompted by AMK's toying with indexing distutils PKG-INFO files. I quickly wrote the PEP, seconded a section of python.org to run it on and hacked together some code to do the job. I specifically limited the scope of the PEP to what I knew I could get done quickly before something more important popped up.

    Running a package download and distribution system, with cryptographic signing and verification procedures, mirroring, etc. Now that's a *lot* of effort.

      posted by Richard Jones at 01:09:41 PM on January 27, 2004  
  12. i will soon write a PEP for python isolates to combat running untrusted code. please implement this as soon as possible. put all your code online and announce it in the newsgroup (c.l.py) in the fasion of release early and often.
      posted by alimoe at 05:05:48 PM on January 28, 2004  
  13. I've had the trust part of the discussion before.
    I do not think that establishing a trust/signing
    system is crucial to an auto-download system,
    since I think most users are already blindly
    downloading anything from any page that looks
    like it might be the Python package they want.

    A 'safe mode', if this auto-installer were to
    have one, would merely have to say "go look
    at this URL. Does the page appear trustworthy?
    [Y/n]". Then it would be at least as good as
    the check that I claim most users do now.
    I also claim that most users would not enable
    such a mode :)


      posted by Drew Perttula at 01:15:22 AM on January 31, 2004