Proposal: new options framework

Rationale:

The options framework has some problems:

 - for a given option we only have two variables: WITH_FOO and WITHOUT_FOO.
   These fail to express some needed abstractions.

 - the fact that their values are not checked (only their existence)
   creates confusion.

 - the usage is not consistent: for instance, if you set a WITH_FOO in
   /etc/make.conf, 'make config' might not check for it and only check
   for WITHOUT_FOO instead.

Proposal:

Here is the proposal and a patch that implements it.

We introduce 3 different types of options: simple, multi and single.

 - simple options are the same as the current options (i.e. on or off.)

 - multi options are options where at least one must be set (1-N).

 - single options are options where one and only one must be set set
   (exclusive options).

A maintainer can define these options in the port Makefile.  Some common
ones will be defined system-wide.

Every option can have an optional description.

A maintainer can set a port's default options (via OPTIONS_DEFAULT)
and override global options that the port doesn't support yet (via
OPTIONS_EXCLUDE).

A user can set global options for all ports in make.conf:

  OPTIONS_SET= OPT1 OPT3

or, similarly, unset them:

  OPTIONS_UNSET= OPT10 OPT15

Per-port options can be specified in two ways for a given port:

 - manually, through /etc/make.conf:

  ${UNIQUENAME}_SET= OPT1
  ${UNIQUENAME}_UNSET= OPT2

   For example, for zsh it would be:

  zsh_OPTIONS_SET= OPT1
  zsh_OPTIONS_UNSET= OPT2

 - or automatically via 'make config' using /var/db/ports/${UNIQUENAME}/options:

  OPTIONS_SET= OPT1
  OPTIONS_UNSET= OPT2

We will supply some generic options descriptions as it is currently done
in the KNOBS file.

In the patch nothing is activated by default; the user has to invoke
check-options manually.

The old framework has been reimplemented on top of the new one.
It should also be easy to extend to add more features later.

Operation:

The framework checks how the options are set using the following
precedence:

  1/ sets the default global options as defined by portmgr;

  2/ removes any global option excluded by the port maintainer via
     OPTIONS_EXCLUDE;

  3/ replaces all the above via OPTIONS_OVERRIDE if set;

  4/ overrides any of the above using the per-port options as defined by
     the maintainer;

  5/ overrides any of the above using the system-wide options (OPTIONS_(UN)SET)
     set by the user in make.conf;

  6/ overrides any of the above using the per-port options set by the user
     in make.conf (${UNIQUENAME:U}_SET and ${UNIQUENAME:U}_UNSET);

  7/ overrides any of the above using the legacy per-port OPTIONS+= in
     /var/db/ports/${UNIQUENAME}/options (note: usage of OPTIONS+= is
     deprecated);

  8/ overrides any of the above using the per-port OPTIONSFILE (as set by
     the dialog UI).

The framework then checks for consistency of the exclusive options and
the 1-n options that are set via 'make check-options'.

Short example:

Here's a Makefile example that includes each type of option:

  OPTIONS_DEFINE= OPT1 OPT2 OPT3

  OPTIONS_MULTI= GRP1 GRP2
  OPTIONS_MULTI_GRP1= OPT4 OPT5
  OPTIONS_MULTI_GRP2= OPT6 OPT7

  OPTIONS_SINGLE= SEL1
  OPTIONS_SINGLE_SEL1= OPT8 OTP9 OPT10

  OPTIONS_DEFAULT= OPT2 OPT3 OPT9 OPT7 OPT8 OPT4
  OPTIONS_EXCLUDE= NLS DOCS

To define a desciption for a given option:

  OPT1_DESC= "Description of my option"

'make showoptions' will present all the options and their descriptions:

  ===> The following configuration options are available:
  OPT1=off: Description of my option
  OPT2=on
  OPT3=on
  ====> Options available for the multi GRP1: you must choose at least one of them
  OPT4=on
  OPT5=off
  ====> Options available for the multi GRP2: you must choose at least one of them
  OPT6=off
  OPT7=on
  ====> Options available for the single SEL1: you must select only one of them
  OPT8=on
  OPT9=off
  OPT10=off

What users need to know:

  OPTIONS_SET = globally set some options
  OPTIONS_UNSET = globally unset some options
  <UNIQUENAME>_SET = set per-port choice of options
  <UNIQUENAME>_UNSET = unset per-port choice of options

OPTIONS_OVERRIDE is to bypass everything and directly set the option
wanted for the ports, whatever is normally set; use with caution.

See above for the precedence.

What maintainers need to know:

For the maintainer to check if an option is set or not, we no longer
need to choose between WITHOUT or WITH options.  Options should be checked
this way:

  if !empty(PORTS_OPTIONS:MOPT1)
  @${ECHO_CMD} " the options OPT1 is set (aka on)"
  .else
  @${ECHO_CMD} "the options OPT1 is not set (aka off)"
  .endif

Variables available to the maintainer are:

  OPTIONS_DEFINE (simple list of options)
  OPTIONS_MULTI (list of the name of option multi)
  OPTIONS_MULTI_<NAME> (list of options for a given multi)
  OPTIONS_SINGLE (list of the name of options single)
  OPTIONS_SINGLE_<NAME> (list of options for a given single)
  OPTIONS_DEFAULT (list of options to be activated by default)

OPTIONS_DEFINE is the list of all options.

For simple (binary) options, if the maintainer does nothing, the option
is off.  However, if the maintainer adds the options to OPTIONS_DEFAULT:

  OPTIONS_DEFINE= A B C D E
  OPTIONS_DEFAULT= A C

then A=on, B=off, C=on, D=off, E=off.

For the other options:

  MULTI = at least 1 but possibly more choices must be made.
  SINGLE = at least 1 and no more than 1 choice must be made.

<NAME>_DESC is a human readable description of the option.

If <NAME> is in OPTIONS_DEFINE then the SINGLE or MULTI concerned is conditionnal.

OPTIONS_EXCLUDE is used to remove an option from the list.  For example,
if DOCS does not mean anything for my port:

  OPTIONS_EXCLUDE+= DOCS

Note: it is not possible to set as off-by-default an option that has been
defined as on-by-default by portmgr (in bsd.options.mk).  But a maintainer
can decide this option does not exists for the given port via OPTIONS_EXCLUDE.

OPTIONS_OVERRIDE can be used to for a port to be built only with the
options given in OPTIONS_OVERRIDE.  (However, check-config will still
be used to force a valid combination of options).

If the maintainer wants to override a description, or set a description
for a custom option OPT1, he just has to define:

   OPT1_DESC= This is my description line

Finally, a compatibility layer allow the old OPTIONS to be valid
declarations, but this usage is deprecated.

What committers need to know:

bsd.options.desc.mk is the central point to define common options and
their default descriptions:

  OPT_DESC?= A cool description

( ?= is important to allow overriding this in per port basis).

What portmgr needs to know:

The first lines of bsd.options.mk contain ALL_OPTIONS, which are options
global to the entire ports tree.

Next are PORT_OPTIONS, which are the options that portmgr claims that
should be on for the whole tree by default:

  PORT_OPTIONS+= <NAME> if portmgr decide an option should be on by default

This forces all ports to have the given set of options (but which can
be ignored by maintainers using OPTIONS_EXCLUDE).

A maintainer has no possibility to unset it by default, but he can still
ignore the option using OPTIONS_EXCLUDE.

XXX MCL disagree: POLA
XXX bapt: what is POLA the inhability to unset by default? it can be easily
XXX change if needed
XXX MCL I disagree that we want NOPORTDOCS by default for POLA.

XXX Bapt: pav don't want us to provide any defaults
XXX Change to match the new values

Notice that in the code, DOCS is used instead of its NO* counterpart
NOPORTDOCS.  The NO* counterparts is still set for compatibility reasons
for the same reasons that WITHOUT_NLS is also defined whenever NLS is 
unset.

By default for compatibility DOCS is set to on and NLS is set to on.

Implementation details:

The options logic is split into 3 files:

 - bsd.options.mk: All the variable definitions concerning global options
   (portmgr territory)

 - bsd.options.desc.mk: Descriptions for common options (accessible to
   all committers)

 - bsd.port.mk: All the code concerning the options framework (portmgr
   territory)

The following new targets are added to bsd.port.mk:

  - check-config: executed automatically, its goal is to check
    the sanity of the options set by the user.

  - pre-check-config and pre-config: used internally to the options
    framework and not intended for users.

  - pretty-print-config: show the options in a brief one-line format
    (e.g.  with no descriptions):

      +OPT for the options that are on;
      -OPT for the options that are off;
      MULTINAME[+OPT -OPT] for the multis;
      SINGLENAME(+OPT -OPT) for the singles.

All the old *config targets (e.g. showconfig) have been reimplemented
on top of the new code.

The dialog now prints all the options +.  The multis are printed this
way: M(multiname): description, and the single: S(singlename): description.

Other changes:

 - we introduce a new global KNOB NO_DIALOG (proposed name) that prevents
   'make config-conditional' from being launched.  dialog remains on by
   default.

An example:

Here is an example of a trivial port Makefile, and what the targets
do in its case:

# cat Makefile

PORTNAME=       test
CATEGORIES=     sysutils

OPTIONS_DEFINE= DOCS NLS A B C D E
OPTIONS_MULTI=A G
OPTIONS_SINGLE=B F
OPTIONS_MULTI_A=M N O
OPTIONS_MULTI_G=P Q
OPTIONS_SINGLE_B=R Z
OPTIONS_SINGLE_F=Y W

.include <bsd.port.mk>

---> check-config

# make
===>  Vulnerability check disabled, database not found
===>  License check disabled, port has not defined LICENSE
====> You should select one and only one options from the F single
*** Error code 1

---> showconfig

# make showconfig
===> The following configuration options are available:
      DOCS=on: Build and install the port documentations
      NLS=on: Build and install the port with localisation
      A=off
      B=off
      C=off
      D=off
      E=off
====> Options available for the multi A: you must choose at least one of them
      M=off
      N=off
      O=off
====> Options available for the multi G: you must choose at least one of them
      P=off
      Q=off
====> Options available for the single B: you must select only one of them
      R=off
      Z=off
====> Options available for the single F: you must select only one of them
      Y=off
      W=off
===> Use 'make config' to modify these settings

---> pretty-print-config

# make pretty-print-config
+DOCS +NLS -A -B -C -D -E A[ -M -N -O ] G[-P -Q ] B( -R -Z ) F( -Y -W )

Another example:

A use case may help: consider

  OPTIONS_DEFINE="A"
  OPTIONS_SINGLE="A"
  OPTIONS_SINGLE_A="B C D"

This means that the A single is conditional on the A option.

If A option is "on" then we will check is the user has correctly set
only one of the following: B C D

This is to have a 0 or 1 among a list option type.

XXX begin roadmap
Things that need to be discussed are:

 - convert KNOBS to Mk/bsd.options.desc.mk

 - slowly modify the ports from knobs/options to optionsng

 - Do we force some options for the whole ports tree aka (ALL_OPTIONS+=
   DOCS etc) or let the maintainer define them?
	* start a portmgr discussion about this (after the patch is in)

 - What options do we want to be default?  FYI we can set default values
   PORT_OPTIONS=BLA even if bla is not in ALL_OPTIONS.
	* start a portmgr discussion about this (after the patch is in)

 - remove the old options compatibility from bsd.port.mk (date?)

 - XXX TODO discuss how we are going to roll this out
XXX end roadmap