Build with pre-built package in Docker Hub #6

- Defaults to using a pre-built docker image in bioconda docker hub.
- Improve naming of scripts and update README with details.
- Test on new version of cutadapt

README.md

@@ -18,30 +18,35 @@ permission.
 We build Linux packages inside a CentOS 5 docker container to maintain
 compatibility across multiple systems. To build for a specific package run:
 
-    docker run --net=host --rm=true -i -t -v `pwd`:/tmp/conda-recipes centos:centos5 /bin/bash /tmp/conda-recipes/update_binstar_packages_docker.sh your_package
+    docker run --net=host --rm=true -i -t -v `pwd`:/tmp/conda-recipes bioconda/bioconda-builder /bin/bash /tmp/conda-recipes/update_packages_docker.sh your_package
 
 Leave off `your_package` to check all of the packages and do builds only if the
-latest is not present in the `bioconda` channel.
+latest is not present in the `bioconda` channel. You can also specify multiple
+packages and force package building or build for multiple platforms. Run
+`update_packages.py --help` for command line options.
 
 To run the Docker script you need to have two files in your `recipes` directory:
 `anaconda-user.txt` and `anaconda-pass.txt` that contain your anaconda.org user
 and password. The build scripts will authenticate with these, but will require a
 manual okay. This is the only non-automated step in the build and upload process.
 
-In some cases, building the docker container can take more time than building
-the conda package. If so, you can build a container locally and use that for
-building instead of the centos:centos5 container. Use the
-`update_binstar_packages_docker_prebuilt.sh` script when using this strategy:
+We use a pre-built CentOS 5 package with compilers installed as part of the
+standard build. To build this yourself and push a new container to
+[Docker Hub](https://hub.docker.com/r/bioconda), you can do:
 
-    (cd docker && docker build -t bioconda-builder .)
-    docker run --net=host --rm=true -i -t -v `pwd`:/tmp/conda-recipes bioconda-builder /bin/bash /tmp/conda-recipes/update_binstar_packages_docker.sh your_package
+    cd docker && docker build -t bicoonda/bioconda-builder .
+    docker push bioconda/bioconda-builder
 
+If you'd like to bootstrap from a bare CentOS and install all
+the packages yourself for testing or development, use the fullbuild script:
+
+    docker run --net=host --rm=true -i -t -v `pwd`:/tmp/conda-recipes centos:centos5 /bin/bash /tmp/conda-recipes/update_packages_docker_fullbuild.sh your_package
 
 ## OSX
 
 For packages that build on OSX, run:
 
-    python update_binstar_packages.py your_package
+    python update_packages.py your_package
 
 or leave off `your_package` to try and build all out of date packages. Packages
 that fail to build on OSX should get added to `CUSTOM_TARGETS` in

cutadapt/meta.yaml

@@ -1,10 +1,10 @@
 package:
   name: cutadapt
-  version: "1.8.1"
+  version: "1.8.3"
 source:
-  fn: cutadapt-1.8.1.tar.gz
-  url: https://pypi.python.org/packages/source/c/cutadapt/cutadapt-1.8.1.tar.gz
-  md5: bcc9cc9d5ac5c8fc5bf91aad42fa7f82
+  fn: cutadapt-1.8.3.tar.gz
+  url: https://pypi.python.org/packages/source/c/cutadapt/cutadapt-1.8.3.tar.gz
+  md5: c7384f3bb834375ad8b0a2869827be6f
 requirements:
   build:
     - python

update_binstar_packages.py → update_packages.py

@@ -17,19 +17,22 @@
     "remote_repo": "bioconda",
     "numpy": "19"}
 
-# Add targets depending on detected current OS
-if sys.platform == "darwin":
-    CONFIG['targets'] = ['osx-64']
-elif sys.platform in ['linux', 'linux2']:
-    CONFIG['targets'] = ['linux-64']
-else:
-    raise ValueError("%s not in [linux, linux2, darwin]: can't determine OS" % sys.platform)
-
 CUSTOM_TARGETS = {
     "hap.py": ["linux-64"]}
 
-def main(pkgs, force_build=True):
+def main(pkgs, force_build=False, multi_platform=False):
     config = CONFIG
+    # Add targets depending on detected current OS
+    if sys.platform == "darwin":
+        config['targets'] = ['osx-64']
+    elif sys.platform in ['linux', 'linux2']:
+        if multi_platform:
+            config["targets"] = ["linux-64", "osx-64"]
+        else:
+            config['targets'] = ['linux-64']
+    else:
+        raise ValueError("%s not in [linux, linux2, darwin]: can't determine OS" % sys.platform)
+
     subprocess.check_call(["conda", "config", "--add", "channels", config["remote_repo"]])
     if not pkgs:
         pkgs = sorted([x for x in os.listdir(os.getcwd()) if os.path.isdir(x)])
@@ -122,5 +125,6 @@ def _should_convert(build_file):
     ap = argparse.ArgumentParser()
     ap.add_argument('packages', nargs="+", help="Package to build")
     ap.add_argument('--force-build', action='store_true', help='Force local build even though the latest version already exists on binstar')
+    ap.add_argument('--multi-platform', action='store_true', help='Try and build for multiple platforms, for python-only packages')
     args = ap.parse_args()
-    main(args.packages, force_build=args.force_build)
+    main(args.packages, force_build=args.force_build, multi_platform=args.multi_platform)

update_binstar_packages_docker_prebuilt.sh → update_packages_docker.sh

@@ -8,4 +8,4 @@ cd /tmp/conda-recipes
 anaconda login --hostname bcbio-conda-auto --username `cat anaconda-user.txt` --password `cat anaconda-pass.txt`
 cd /tmp/conda-recipes
 export PATH=/tmp/conda-build/anaconda/bin:$PATH
-/tmp/conda-build/anaconda/bin/python update_binstar_packages.py $*
+/tmp/conda-build/anaconda/bin/python update_packages.py $*

update_binstar_packages_docker.sh → update_packages_docker_fullbuild.sh

@@ -38,4 +38,4 @@ anaconda login --hostname bcbio-conda-auto --username `cat anaconda-user.txt` --
 
 cd /tmp/conda-recipes
 export PATH=/tmp/conda-build/anaconda/bin:$PATH
-/tmp/conda-build/anaconda/bin/python update_binstar_packages.py "$*"
+/tmp/conda-build/anaconda/bin/python update_packages.py "$*"