This patch replaces do-fetch/checksum/makesum/fetch-list targets in 
bsd.port.mk with a shell script, eliminating more than 1/2k lines of
code. The primary focus is to break down bsd.port.mk into components,
in order to make refactoring posssible. 

As a side effect, time critical operations, such as INDEX builds and
port surveys should run a little faster, while distfiles handling
operations like fetching and extracting are mostly not affected by
make running times.

In case you need to fall back to the old routines, just say
    make OLD_DISTFILES_HANDLING=yes ...

Added features are:
- sort MASTER_SITES by table lookup (PR 63933, use distrank.sh to create a sample table)
- refetch from multiple mirrors in case of a checksum mismatch (up to FETCH_REGET times)
- randomize MASTER_SITES by default (disable with KEEP_MASTER_SITE_ORDER)
- new target `missing-size-recursive':  calculate size of missing distribution files (PR 62271)
- new target checkdistfiles, a brute-force distfile survery for home use (PR 64157),
  use with NO_CHECKSUM to just verify the sizes without downloading the files
- support for integrating distinfo into the ports Makefile via `migratesum' (experimental,
  multiple integrated distinfo files could be distinguised by DISTINFO_LABEL)
- support for SHA1 and RMD160 when OpenSSL is present (PR 56641)
- support for MASTER_SITES that don't end with a slash
- support for %FILE% in MASTER_SITES
- fetch with curl or wget with FETCH_USE_CURL and FETCH_USE_WGET

How to use:

cd /usr/ports
fetch -o - http://people.freebsd.org/~eik/patches/distfiles/bsd.port.mk.patch | patch
fetch -o Tools/scripts/distfiles.sh http://people.freebsd.org/~eik/patches/distfiles/distfiles.sh


Feedback / patches welcome.

-Oliver



Known issues:

The ports

  devel/psvn
  devel/ruby-cvs
  devel/ruby-tzfile
  net/hf6to4
  x11-themes/kde-icons-icosx

don't fetch. A work around is to do

  sed -i '.bak' -E '/^DISTNAME=/s/\\//g' Makefile

or apply the following patch to net/hf6to4:

--- Makefile	8 Jan 2004 23:56:22 -0000	1.2
+++ Makefile	29 May 2004 15:15:48 -0000
@@ -9,9 +9,9 @@
 PORTVERSION=	1.2
 CATEGORIES=	net ipv6
 MASTER_SITES=	http://cvsweb.netbsd.org/bsdweb.cgi/~checkout~/pkgsrc/net/hf6to4/files/
-DISTFILES=	hf6to4\?rev=1.3\&content-type=text%2fplain \
-		hf6to4.8\?rev=1.2\&content-type=text%2fplain \
-		hf6to4.conf\?rev=1.2\&content-type=text%2fplain
+DISTFILES=	hf6to4?rev=1.3&content-type=text%2fplain \
+		hf6to4.8?rev=1.2&content-type=text%2fplain \
+		hf6to4.conf?rev=1.2&content-type=text%2fplain
 DIST_SUBDIR=	${PORTNAME}
 
 MAINTAINER=	fuyuki@nigredo.org
@@ -25,7 +25,7 @@
 do-extract:
 	${MKDIR} ${WRKSRC}
 .for i in ${DISTFILES}
-	${CP} ${DISTDIR}/${DIST_SUBDIR}/${i} ${WRKSRC}/${i:C/\?.*$//}
+	${CP} "${DISTDIR}/${DIST_SUBDIR}/${i}" ${WRKSRC}/${i:C/\?.*$//}
 .endfor