Received: from SU-DSN by SU-AI with PUP; 18-May-83 23:13 PDT Received: From MIT-XX by SU-DSN.ARPA; Wed May 18 23:15:14 1983 Received: from SCRC-EUPHRATES by SCRC-SPANIEL with CHAOS; Thu 19-May-83 02:14:51-EDT Date: Thursday, 19 May 1983, 02:14-EDT From: David A. Moon Subject: error proposal not so hard to implement To: Common-Lisp at su-ai I implemented everything in my error proposal this evening, including hairy interface to the debugger. It took about five hours.  Received: from CMU-CS-C by SU-AI with TCP/SMTP; 18 May 83 21:46:45 PDT Received: ID ; 19 May 83 00:47:40 EDT Date: Thu, 19 May 1983 00:47 EDT From: Scott E. Fahlman To: common-lisp@su-ai Subject: Revenge of the Packages Minor cleanup and incorporation of Moon's treatise on name conflicts. *************************************************************************** @chapter [Packages] @section [Overview] One problem with earlier Lisp systems is the use of a single name space for all symbols. In large Lisp systems, with modules written by many different programmers, accidental name collisions become a serious problem. Common Lisp addresses this problem through the @i[package system], derived from an earlier package system developed for Lisp Machine Lisp. In addition to preventing name-space conflicts, the package system makes the modular structure of large Lisp systems more explicit. A @i[package] is a data structure that establishes a mapping from print names (strings) to symbols. The package thus replaces the ``oblist'' or ``obarray'' machinery of earlier Lisp systems. At any given time one package is current, and this package is used by the reader in translating strings into symbols. The current package is always the value of the global variable @b[*package*]. It is possible to refer to symbols in packages other than the current one through the use of @i[package qualifiers] in the symbol reference. For example @b[foo:bar] refers to the symbol whose name is @b[bar] in the package whose name is @b[foo]. The string-to-symbol mappings available in a given package are divided into two classes, @i[external] and @i[internal]. We refer to the symbols accessible via these mappings as being @i[external] and @i[internal] symbols of the package in question, though really it is the mappings that are different and not the symbols themselves. Within a given package, a name refers to one symbol or to none; if it does refer to a symbol, that symbol is either external or internal in that package, but not both. External symbols are part of the package's public interface to other packages. These are supposed to be chosen with some care and are advertised to users of the package. Internal symbols are for internal use only, and these symbols are normally hidden from other packages. Most symbols are created as internal symbols; they become external only if they appear explicitly in some package's @b[export] command. A symbol may appear in many packages. It will always have the same print name wherever it appears, but it may be external in one package and internal in another. A name may refer to different symbols in different packages. Packages may be built up in layers. From the point of view of a package's user, the package is a single collection of mappings from strings into internal and external symbols. However, some of these mappings may be established within the package itself, while other mappings are inherited from other packages via the @b[use-package] construct. (The mechansims responsible for this inheritance are described below.) In what follows, we will refer to a symbol as being "accessible" in a package if it can be referred to in that package without a qualifier, regardless of whether the mapping occurs within that package or via inheirtance; we will refer to a symbol as being "present" in a package if the mapping is in the package itself and is not inherited from somewhere else. @section [Consistency Rules] Package-related bugs can be very subtle and confusing: things are not what they appear to be. The Common Lisp package system is designed with a number of safety features to prevent most of the common bugs that would otherwise occur in normal use. This may seem over-protective, but experience with earlier package systems has shown that such safety features are needed. In dealing with the package system, it is useful to keep in mind the following consistency rules, which remain in force as long as the value of @b[*package*] is not changed by the user or his code: @begin [itemize] @b[Read-Read consistency:] Reading the same print name always gets you the same (EQ) symbol. @b[Print-Read consistency:] An interned symbol always prints as a sequence of characters which, when read back in, yields the same (EQ) symbol. @b[Print-Print consistency:] If two interned symbols are not EQ, then their printed representations will not be the same sequence of characters. @end [itemize] These consistency rules remain true in spite of any amount of implicit interning caused by typing in Lisp forms, loading files, etc. This has the important implication that results are reproducible regardless of the order of loading files or the exact history of what symbols were typed in when. The rules can only be violated by explicit action: changing the value of @b[*package*], forcing some action by continuing from an error, or calling one of the ``dangerous'' functions: @b[unintern], @b[shadow], or @b[shadowing-import]. @section [Package Names] Each package has a name (a string) and perhaps some nicknames. These are assigned when the package is created, though they can be changed later. A package's name should be something long and self-explanatory like "editor"; there might be a nickname that is shorter and easier to type, like "ed". Package name strings retain whatever mixture of upper and lower case characters the user supplies, but the translation from names to packages is case-insensitive, so users can generally refer to packages without worrying about case. There is a single name space for packages. The function @b[find-package] translates a package-name or nickname into the associated package. The function @b[package-name] returns the name of a package. The function @b[package-nicknames] returns a list of all nicknames for a package. The function @b[rename-package] removes a package's current name and nicknames and replaces them with new ones specified by the user. Package renaming is occasionally useful when, for development purposes, it is desirable to load two versions of a package into the same Lisp. We can load one, rename it, and then load the other, without getting a lot of name conflicts. @section [Translating Strings to Symbols] The value of the special variable @b[*package*] must always be a package object (not a name). This is referred to as the @i[current package]. When the Lisp reader has, by parsing, obtained a string of characters thought to name a symbol, that name is looked up in the current package. This lookup may involve looking in other packages whose external symbols are inherited by the current package (see below). If the name is found, the corresponding symbol is returned. If the name is not found, a new symbol is created for it and is placed in the current package as an internal symbol; if the name is seen again while this same package is current, the same symbol will then be returned. When a new symbol is created, a pointer to the package in which it is initially placed is stored in the @i[package cell] of that symbol; the package is said to be the symbol's @i[home package], and is said to @i[own] the symbol. (Some symbols are not owned by any package; they are said to be @i[uninterned], and their package cell contains NIL.) Often it is desirable to refer to an external symbol in some package other than the current one. This is done through the use of a @i[qualified name], consisting of a package name, then a colon, then the print name of the symbol. This causes the symbol's name to be looked up in the specified package, rather than in the current one. For example, ``@b[editor:buffer]'' refers to the external symbol named ``@b[buffer]'' available in the package named ``@b[editor]'', regardless of whether there is a symbol named ``@b[buffer]'' in the current package. If there is no package named ``@b[editor]'', or if no symbol named ``@b[buffer]'' is available in ``@b[editor]'' or if ``@b[buffer]'' is an internal symbol in ``@b[editor]'', a correctable error is signalled to ask the user what he really wants to do. On rare occasions, a user may need to refer to an @b[internal] symbol of some package other than the current one. It is illegal to do this with the colon qualifier, since accessing an internal symbol of some other package is usually a mistake. However, this operation is legal if you use ``@b[#:]'' as the separator in place of the usual colon. If ``@b[editor#:buffer]'' is seen, the effect is exactly the same as reading ``@b[buffer]'' with @b[*package*] temporarily rebound to the package whose name is ``@b[editor]''. This special-purpose qualifier should be used with caution. The package named @b[keyword] contains all keyword symbols used by the Lisp system itself and by user-written code. Such symbols must be easily accessible from any package, and name conflicts are not an issue since these symbols are used only to label arguments and never to carry package-specific values or properties. Because keyword symbols are used so frequently, Common Lisp provides a special reader syntax for them: any symbol preceded by a colon but no package name (for example ``@b[:foo]'') is added to (or looked up in) the @b[keyword] package as an @i[external] symbol. The @b[keyword] package is also treated specially in that whenever a symbol is added to the @b[keyword] package, the symbol is automatically declared to be a constant and is made to have itself as its value. This is why every keyword evaluates to itself. As a matter of style, keywords should always be accessed using the leading-colon convention; you never want to import or inherit keywords into any other package. Each symbol contains a package slot that is used to record the home package of the symbol, or NIL if the symbol is uninterned. This slot may be accessed by @b[symbol-package] @ref[%%%]. When an interned symbol is printed, if it is an external symbol in the keyword package then it is printed with a preceding colon; otherwise, if it is available (directly or by inheritance) in the current package, it is printed without any qualification; otherwise, it is printed with the name of the home package as the qualifier, using ``@b[:]'' as the separator if the symbol is external and ``@b[#:]'' if not. A symbol that is uninterned (has no home package) is printed preceded by ``@b[#:]''. It is possible, by the clever use of import and unintern, to create a symbol that has no recorded home package, but that in fact is interned in some package. The system does not check for this. In summary, the following four cases are defined: @Begin[describe] @b[foo:bar] @\Look up "bar" among the external symbols available in the package named "foo". @b[foo#:bar] @\Look up "bar" among the external and internal symbols available in the package named "foo". @b[:bar] @\The symbol named "bar" is a self-evaluating keyword. @b[#:bar] @\The symbol named "bar" is uninterned. @End[describe] All other uses of colons within names of symbols are not defined by Common Lisp, but are reserved for implementation-dependent use; this includes names that end in a colon, contain two or more colons, or consist of just a colon. @section [Exporting and Importing Symbols] Symbols from one package may be made available in another package in two ways. First, any individual symbol may be added to a package by use of the @b[import] function. The form @b[(import 'editor:buffer)] takes the external symbol named @b[buffer] in the @b[editor] package (this symbol was located when the form was read by the Lisp reader) and adds it to the current package as an internal symbol. The imported symbol is not automatically exported from the current package, but if it is already present and external, that is not changed. After the call to @i[import] it is possible to refer to @b[buffer] in the importing package without any qualifier. The status of @b[buffer] in the package named @b[editor] is unchanged, and @b[editor] remains the home package for this symbol. Once imported, a symbol is @i[present] in the importing package and can only be removed by calling @b[unintern]. If the symbol is already present in the importing package, @b[import] has no effect. If a distinct symbol with the name @b[buffer] is available in the importing package (directly or by inheritance) a correctable error is signalled, as described in the next section. If the user really wants to do a shadowing import without getting an error, he should use the @b[shadowing-import] function. This inserts the symbol into the specified package as an internal symbol, regardless of whether another symbol of the same name will be shadowed by this action. If the package already contains a different symbol of the same name, this will be uninterned from that package. The new symbol is added to the package's shadowing-symbols list. @b[Shadowing-import] should be used with caution. It changes the state of the package system in such a way that the consistency rules do not hold across the change. The second mechanism is provided by the @b[use-package] function. This causes a package to inherit all of the external symbols of some other package. These symbols become available as @i[internal] symbols of the using package. That is, they can be referred to without a qualifier while this package is current, but they are not passed along to any other package that uses this package. Use-package checks carefully for name conflicts between the newly imported symbols and those already available in the importing package. This is described in detail in the next section. Often a user, working by default in the @b[user] package, will load a number of packages into his Lisp to provide an augmented working environment; then he will call @b[use-package] on each of these packages so that he can easily access their external symbols. @b[Unuse-package] undoes the effects of a previous @b[use-package]. The external symbols of the used package are no longer inherited. However, any symbols that have been imported into the using package continue to be present in that package. There is no way to inherit the @i[internal] symbols of another package; to refer to an internal symbol, you must either make that symbol's home package current, use a qualifier, or import that symbol into the current package. When @b[intern] or some other function wants to look up a symbol in a given package, it first looks for the symbol among the external and internal symbols of the package itself; then it looks through the external symbols of the used packages in some unspecified order. The order does not matter; according to the rules for handling name conflicts (see below), if conflicting symbols appear in two or more packages inherited by package X, a symbol of this name must also appear in X itself as a shadowing symbol. Of course, implementations are free to choose other, more efficient ways of implementing this search, as long as the user-visible behavior is equivalent to what is described here. The @b[export] function takes a symbol that is available in some package and makes it an external symbol of that package. If the symbol is already available as an external symbol in the package, @b[export] has no effect. If the symbol is directly present in the package as an internal symbol, it is simply changed to external status. If it is available as an internal symbol via @b[use-package], the symbol is first imported into the package, then exported. (It is now present in this package whether or not it continues to use the symbol's original package.) If the symbol is not available at all in the specified package, a correctable error is signalled which, upon continuing, asks the user whether the symbol should be imported. The @b[unexport] function is provided mainly as a way to undo erroneous calls to @b[export]. It works only on symbols that are directly present in the current package, switching them back to internal status. If @b[unexport] is given a symbol that is already available as an internal symbol in the current package, it does nothing; if it is given a symbol that is not available in the package at all, it signals an error. @section[Name Conflicts] A fundamental invariant of the package system is that within one package any particular name can only refer to one symbol. A @i[name conflict] is said to occur when there is more than one candidate symbol and it is not obvious which one to choose. If the system does not always choose the same way, the read-read consistency rule would be violated. For example, some programs or data might have been read in under a certain mapping of the name to a symbol. If the mapping changes to a different symbol, then additional programs or data are read, the two programs will not access the same symbol even though they use the same name. Even if the system did always choose the same way, a name conflict is likely to result in a different mapping from names to symbols than was expected by the user, causing programs to execute incorrectly. Therefore, any time a name conflict occurs, an error is signalled. The user may continue from the error and tell the package system how to resolve the conflict. Note that if the same symbol is accessible to a package through more than one path, for instance as an external of more than one package, or both through inheritance and through direct presence in the package, there is no name conflict. Name conflicts only occur between distinct symbols with the same name. The creator of a package can tell the system in advance how to resolve a name conflict through the use of @i[shadowing]. Every package has a list of shadowing symbols. A shadowing symbol takes precedence over any other symbol of the same name that would otherwise be accessible to the package. A name conflict involving a shadowing symbol is always resolved in favor of the shadowing symbol, without signalling an error (except for one exception involving @b[import] described below). The functions @b[shadow] and @b[shadowing-import] may be used to declare shadowing symbols. Name conflicts are detected when they become possible, i.e. when the package structure is altered. There is no need to check for name conflicts during every name lookup. The functions @b[use-package], @b[import], and @b[export] check for name conflicts. @b[use-package] makes the external symbols of the package being used accessible to the using package; each of these symbols is checked for name conflicts with the symbols already accessible. @b[import] adds a single symbol to the internals of a package, checking for a name conflict with an existing symbol either present in the package or accessible to it. @b[import] signals an error even if there is a name conflict with a shadowing symbol, because two explicit directives from the user are inconsistent. @b[export] makes a single symbol accessible to all the packages that use the package from which the symbol is exported. All of these packages are checked for name conflicts: @b[(export @i[s] @i[p])] does @b[(find-symbol (symbol-name @i[s]) @i[q])] for each package @i[q] in @b[(package-used-by-list @i[p])]. Note that in the usual case of an @b[export] during the initial definition of a package, the @b[package-used-by-list] will be @b[nil] and the name conflict checking will take no time. @b[intern] does not need to do any name-conflict checking, because it never creates a new symbol if there is already an accessible symbol with the name given. @b[shadow] and @b[shadowing-import] never signal a name-conflict error, because by calling these functions the user has specified how any possible conflict is to be resolved. @b[shadow] does name-conflict checking to the extent that it checks whether a distinct existing symbol with the specified name is accessible, and if so whether it is directly present in the package or inherited; in the latter case a new symbol is created to shadow it. @b[shadowing-import] does name-conflict checking to the extent that it checks whether a distinct existing symbol with the same name is accessible; if so it is shadowed by the new symbol, which implies that it must be @b[unintern]ed if it was directly present in the package. @b[unuse-package], @b[unexport], and @b[unintern] (when the symbol being @b[unintern]ed is not a shadowing symbol) do not need to do any name-conflict checking, because they only remove symbols from a package; they do not make any new symbols accessible. @b[unintern] of a shadowing symbol can uncover a name conflict that had previously been resolved by the shadowing. If package A uses packages B and C, A contains a shadowing symbol @b[x], and B and C each contain external symbols named @b[x], then @b[unintern]ing @b[x] from A will reveal a name conflict between @b[b:x] and @b[c:x] if those two symbols are distinct. In this case @b[unintern] will signal an error. Aborting from a name-conflict error leaves the original symbol accessible. Package functions always signal name-conflict errors before making any change to the package structure. Note: when multiple changes are to be made, for example when @b[export] is given a list of symbols, it is legal for each change to be processed separately, so that aborting from a name conflict caused by the second symbol in the list will not unexport the first symbol in the list. However, aborting from a name conflict error caused by @b[export] of a single symbol will not leave that symbol accessible to some packages and inaccessible to others; @b[export] appears as an atomic operation. Continuing from a name-conflict error should offer the user a chance to resolve the name conflict in favor of either of the candidates. The package structure should be altered to reflect the resolution of the name conflict, via @b[shadowing-import], @b[unintern], or @b[unexport]. A name conflict in @b[use-package] between a symbol directly present in the using package and an external symbol of the used package may be resolved in favor of the first symbol by making it a shadowing symbol, or in favor of the second symbol by @b[unintern]ing the first symbol from the using package. The latter resolution is dangerous if the symbol to be @b[unintern]ed is an external symbol of the using package since it will cease to be an external symbol. A name conflict in @b[use-package] between two external symbols inherited by the using package from other packages may be resolved in favor of either symbol by importing it into the using package and making it a shadowing symbol. A name conflict in @b[export] between the symbol being exported and a symbol already present in a package that would inherit the newly-exported symbol may be resolved in favor of the exported symbol by @b[unintern]ing the other one, or in favor of the already-present symbol by making it a shadowing symbol. A name conflict in @b[export] or @b[unintern] due to a package inheriting two distinct symbols with the same name from two other packages may be resolved in favor of either symbol by importing it into the using package and making it a shadowing symbol, just as with @b[use-package]. A name conflict in @b[import] between the symbol being imported and a symbol inherited from some other package may be resolved in favor of the symbol being imported by making it a shadowing symbol, or in favor of the symbol already accessible by not doing the @b[import]. A name conflict in @b[import] with a symbol already present in the package may be resolved by @b[unintern]ing that symbol, or by not doing the @b[import]. Good user-interface style dictates that @b[use-package] and @b[export], which can cause many name conflicts simultaneously, first check for all of the name conflicts before presenting any of them to the user. The user may then choose to resolve all of them wholesale, or to resolve each of them individually, requiring a lot of interaction but permitting different conflicts to be resolved different ways. Some implementations will have other ways of resolving name conflicts to offer. For instance, if the symbols that conflict are not being used as objects, but only as names for functions, it may be possible to "merge" the two symbols by putting the function definition onto both symbols. References to either symbol for purposes of calling a function would be equivalent. A similar merging operation can be done for variable values and for things stored on the property list. On the Lisp machine one can also @i[forward] the value, function, and property cells so that future changes to either symbol will propagate to the other one. Some other implementations are able to do this with value cells, but not with property lists. Only the user can know whether this way of resolving a name conflict is adequate, i.e. the fact that there are two non-@b[eq] symbols with the same name doesn't affect the correct operation of his program. The value of offering symbol-merging as a way of resolving name conflicts is that it can avoid the need to throw away the whole Lisp world and start over, after correcting the package-definition forms that caused the error. @Section[Built-in Packages] The following packages are built into the Common Lisp system: @begin[description] @b[lisp]@\The package named @b[lisp] contains the primitives of the Common Lisp system. Its external symbols include all of the user-visible functions and global variables that are present in the Common Lisp system, such as @b[car], @b[cdr], @b[*package*], etc. Almost all other packages will want to use @b[lisp] so that these symbols are available without qualification. @b[user]@\The @b[user] package is, by default, the current package at the time a Common Lisp system starts up. This package uses the @b[lisp] package. @b[keyword]@\This package contains all of the keywords used by built-in or user-defined Lisp functions. Symbols that start with a colon are treated as external symbols in the this package. All symbols in this package are treated as constants that evaluate to themselves, so the user can type @b[:foo] instead of @b[':foo]. @b[system]@\This package name is reserved to the implementation. Normally this is used to contain names of implementation-dependent system-interface functions. This package uses @b[lisp] and has the nickname @b[sys]. @end[description] @Section[Package System Functions and Variables] Some of the following have been described earlier, but are included here for completeness. It is up to each implementation's compiler to ensure that when a compiled file is loaded, all of the symbols in the file end up in the same packages that they would occupy if the Lisp source file were loaded. In most compilers, this will be accomplished by treating certain package operations as though they are surrounded by @b[(eval-when (compile load) ...). These operations are @b[make-package], @b[in-package], @b[import], @b[export], @b[unexport], @b[shadow], @b[shadowing-import], @b[use-package], and @b[unuse-package]. To guarantee proper compilation in all Common Lisp implementations, these functions should appear only at top-level within a file. As a matter of style, it is suggested that each file contain only one package, and that all of the package setup forms appear near the start of the file. @begin [implementation-note] In the past, some Lisp compilers have read the entire file into Lisp before processing any of the forms. Other compilers have arranged for the loader to do all of its interns before evaluating any of the top-level forms. Neither of these techniques will work in a straightforward way in Common Lisp because of the presence of multiple packages. @end [implementation-note] For the functions described here, all optional arguments named @i[package] default to the current value of @b[*package*]. Where a function takes an argument that is either a symbol or a list of symbols, an argument of NIL is treated as an empty list of symbols. @Defvar[Var {*package*}] The value of this variable must be a package; this package is said to be the current package. The initial value of @b[*package*] is the @b[user] package. The @b[load] function rebinds @b[*package*] to its current value. If some form in the file changes the value of @b[*package*] during loading, the old value will be restored when the loading is completed. @Enddefvar @Defun[Fun {make-package}, Args {@i[package-name] @key @i[nicknames] @i[use]}] Creates and returns a new package with the specified package name. The @b[:nicknames] argument must be a list of strings to be used as alternative names for the function. These names must not conflict with any existing package names; if they do, a correctable error is signalled. The @b[:use] argument is a list of packages or package names whose external symbols are to be inherited by the new package. These package must already exist. If not supplied, @i[:use] defaults to a list of one package, the @b[lisp] package. @Enddefun @Defun[Fun {in-package}, Args {@i[package-name] @key @i[nicknames] @i[use]}] The @b[in-package] function is intended to be placed at the start of a file containing a subsystem that is to be loaded into some package other than @b[user]. If there is not already a package named @i[package-name], this function is similar to @b[make-package], except that after the new package is created, @b[*package*] is set to it. This binding will remain in force until changed by the user (perhaps with another @b[in-package] call), or until the @b[*package*] variable reverts to its old value at the completion of a @b[load] operation. If there is an existing package whose name is @i[package-name], the assumption is that the user is re-loading a file after making some changes. The existing package is augmented to reflect any new nicknames or new packages in the @b[:use] list (with the usual error-checking) and @b[*package*] is then set to this package. @enddefun @Defun[Fun {find-package}, Args {@i[name]}] The @i[name] must be a string that is the name or nickname for a package. Returns the corresponding package, or NIL if no such package exists. This match ignores case (as in @b[string-equal]). @Enddefun @Defun[Fun {package-name}, Args {@i[package]}] The argument must be a package. This function returns the string that names that package. @Enddefun @Defun[Fun {package-nicknames}, Args {@i[package]}] The argument must be a package. This function returns the list of nickname strings for that package, not including the primary name. @Enddefun @Defun[Fun {package-use-list}, Args {@i[package]}] Returns the list of other packages used by the argument package. @Enddefun @Defun[Fun {package-used-by-list}, Args {@i[package]}] Returns the list of other packages that use the argument package. @Enddefun @Defun[Fun {package-shadowing-symbols}, Args {@i[package]}] Returns the list of symbols in that have been declared as shadowing symbols in this package by @b[shadow] or @b[shadowing-import]. All symbols on this list are present in the specified package. @Enddefun @Defun[Fun {list-all-packages}, Args {}] This function returns a list of all packages that currently exist in the Lisp system. @Enddefun @Defun[Fun {intern}, Args {@i[string] @optional @i[package]}] @i[Package], which defaults to the current package, is searched for a symbol with the name specified by the @b[string] argument. This search will include inherited symbols, as described above. If a symbol with the specified name is found, it is returned. If no such symbol is found, one is created and is installed in the current package as an internal symbol. This symbol is returned. @Incompatibility{Conceptually, @b[intern] translates a string to a symbol. In Maclisp and several other Lisps, @b[intern] can take either a string or a symbol as its argument; in the latter case, the symbol's print name is extracted and used as the string. However, this leads to some confusing issues about what to do if @b[intern] finds a symbol that is not @b[eq] to the argument symbol. To avoid such confusion, we require the argument to be a string.} @Enddefun @Defun[Fun {find-symbol}, Args {@i[string] @optional @i[package]}] This is identical to @b[intern], but it never creates a new symbol. If a symbol with the specified name is found in the current package, directly or by inheritance, the symbol found is returned. The second return value is T, indicating that the symbol was found. The third value is T if the symbol is an external symbol in the specified package, NIL if it is internal. If the symbol is not found in the specified package, @b[find-symbol] returns NIL for all three values. @Enddefun @Defun[Fun {find-external-symbol}, Args {@i[string] @optional @i[package]}] This is identical to @b[find-symbol], but it does not look for internal symbols even in @i[package] itself. This is the type of search done by the reader for symbols qualified with a package name and a colon. The return values are the same as for @b[find-symbol], with the third return value always being T (external symbol) if a symbol is found. @Enddefun @Defun[Fun {unintern}, Args {@i[symbol] @optional @i[package]}] If the specified symbol is present in the specified package, it is removed from this package, and also from the package's shadowing-symbols list if it is present there. Moreover, if @i[package] is the home package for the symbol, the symbol is made to have no home package. @b[Unintern] returns T if it actually removed a symbol, and NIL otherwise. @b[Unintern] should be used with caution. It changes the state of the package system in such a way that the consistency rules do not hold across the change. @Incompatibility{The equivalent of this in @maclisp is @f[remob].} @Enddefun @Defun[Fun {export}, Args {@i[symbols] @optional @i[package]}] The @i[symbols] argument should be a list of symbols, or possibly a single symbol. These symbols become available as external symbols in the current package. (See the sections above for details.) Returns T. By convention, a call to @f[export] listing all exported symbols is placed near the start of a file to advertise which of the symbols mentioned the file are intended to be used by other programs. @Enddefun @Defun[Fun {unexport}, Args {@i[symbols] @optional @i[package]}] The argument should be a list of symbols, or possibly a single symbol. These symbols become internal symbols in @i[package]. (See the sections above for details.) Returns T. @Enddefun @Defun[Fun {import}, Args {@i[symbols] @optional @i[package]}] The argument should be a list of symbols, or possibly a single symbol. These symbols become internal symbols in @i[package], and can therefore be referred to without a colon qualifier. @b[Import] signals a correctable error if any of the imported symbols has the same name as some distinct symbol already available in the package. (See the sections above for details.) Returns T. @Enddefun @Defun[Fun {shadowing-import}, Args {@i[symbols] @optional @i[package]}] This is like import, but it does not signal an error even if the importation of a symbol would shadow some symbol already available in the package. In addition to being imported, the symbol is placed on the shadowing-symbols list of @i[package]. (See the sections above for details.) @b[Shadowing-import] should be used with caution. It changes the state of the package system in such a way that the consistency rules do not hold across the change. Returns T. @Enddefun @Defun[Fun {shadow}, Args {@i[symbols] @optional @i[package]}] The argument should be a list of symbols, or possibly a single symbol. The print-name of each symbol is extracted, and the current package is searched for a symbol of that name. If such a symbol is present in this package (directly, not by inheritance) then nothing is done. Otherwise, a new symbol is created with this print name, and it is inserted in the current package as an internal symbol. The symbol is also placed on the shadowing-symbols list of @i[package]. (See the sections above for details.) @b[Shadow] should be used with caution. It changes the state of the package system in such a way that the consistency rules do not hold across the change. Returns T. @Enddefun @Defun[Fun {use-package}, Args {@i[packages-to-use] @optional @i[package]}] The @i[packages-to-use] argument should be a list of packages or package names, or possibly a single package or package name. These packages are added to the use-list of @i[package] if they are not there already. All external symbols in the packages to use become available in @i[package] as internal symbols. (See the sections above for details.) Returns T. @enddefun @Defun[Fun {unuse-package}, Args {@i[packages-to-unuse]}] The @i[packages-to-unuse] argument should be a list of packages or package names, or possibly a single package or package name. These packages are removed from the use-list of @i[package]. Returns T. @enddefun @Defun[Fun {find-all-symbols}, Args {@i[string-or-symbol]}] Searches every package in the Lisp system for symbols whose print-name is the specified string, and returns a list of such symbols. This search is case-sensitive. If the argument is a symbol, its print-name supplies the string to be searched for. @enddefun @Defmac[Fun {do-symbols}, Args {(@i[var] @Mopt<@i[package]> @Mopt<@i[result-form]>) @Mstar<@i[declaration]> @mstar<@i[tag] @mor @i[statement]>}] @b[Do-symbols] provides straightforward iteration over the symbols of a package. The body is performed once for each symbol available in the @i[package], in no particular order, with the variable @i[var] bound to the symbol. Then @i[resultform] (a single form, @i[not] an implicit @b[progn]) is evaluated, and the result is the value of the @b[do-symbols] form. (When the @i[resultform] is evaluated, the control variable @i[var] is still bound, and has the value @nil.) If @i[resultform] is omitted, the result is NIL. @b[Return] may be used to terminate the iteration prematurely. If execution of the body affects which symbols are contained in the @i[package], other than possibly to remove the symbol currently the value of @i[var] by using @b[unintern], the effects are unpredictable. @Enddefmac @Defmac[Fun {do-external-symbols}, Args {(@i[var] @Mopt<@i[package]> @Mopt<@i[result-form]>) @Mstar<@i[declaration]> @mstar<@i[tag] @mor @i[statement]>}] @b[Do-external-symbols] is just like @b[do-symbols], except that only the external symbols of the specified package are scanned. @Enddefmac @Defmac[Fun {do-all-symbols}, Args {(@i[var] @Mopt<@i[result-form]>) @Mstar<@i[declaration]> @mstar<@i[tag] @mor @i[statement]>}] This is similar to @b[do-symbols], but executes the body once for every symbol contained in every package. (This may not get all symbols whatsoever, depending on the implementation.) It is @i[not] in general the case that each symbol is processed only once, since a symbol may appear in many packages. @Enddefmac @Section[Modules] A @i[module] is a Common Lisp subsystem that is loaded from one or more files. A module is normally loaded as a single unit, regardless of how many files are involved. A module may consist of one package or several packages. The file-loading process is necessarily implementation-dependent, but Common Lisp provides some very simple portable machinery for naming modules, keeping track of which modules have been loaded, and for loading modules as a unit. @Defvar[Var {*modules*}] @Defun[Fun {provide}, Args {@i[module-name]}] @Defun1[Fun {require}, Args {@i[module-name] @optional @i[pathname]}] Each module has a unique name (a string). If the module consists of a single package, it is customary for the package and module names to be the same. The variable @b[*modules*] is a list of names of the modules that have been loaded into the Lisp system so far. The @b[provide] function adds a new module name to this list, thereby indicating that the module in question has been loaded. The @b[require] function tests whether a module is already present (using a case-insensitive comparison) and, if not, loads the appropriate file or set of files. The pathname argument, if present, is a single pathname or a list of pathenames whose files are to be loaded in order, left to right. If the pathname argument is NIL or is not provided, the system will attempt to determine which files to load in some system-dependent manner. This will typically involve some central registry of module names and the associated file-lists. @section [An Example] Most users will want to load and use packages but will never need to build one. Often, a user will load a number of packages into the @b[user] package whenever he uses Common Lisp. Most implementations will provide some sort of "init file" mechanism to make such setup automatic when the Lisp starts up. @begin [Lisp] ;;; Lisp init file for I. Newton. ;;; Set up the USER package the way I like it. ;;; Calculus is used a lot. Get easy access to all its symbols. (require "calculus") (use-package "calculus") ;;; Same for Newtonian mechanics. (require "newtonian-mechanics") (use-package "newtonian-mechanics") ;;; I just want a few thing from here, and other things conflict. ;;; Import just what I need into the USER package. (require "relativity") (import '(relativity:speed-of-light relativity:ignore-small-errors)) ;;; These are worth loading, but I will use qualifiers to get at any ;;; symbols I need. (require "phlogiston") (require "alchemy") @end [Lisp] When each of two files uses some symbols from the other, we must be careful to put the contents of the file in the file in the proper order. Typically each file contains a single package that is a complete module. The contents of such a file should include the following items, in order: @Begin[enumerate] A @b[provide] call that announces the module name. An @b[in-package] call that establishes the package. A @b[shadow] call that establishes any local symbols that will shadow symbols that would otherwise be inherited from packages that this package will use. An @b[export] call that establishes all of this package's external symbols. Any number of @b[require] calls to load other modules which the contents of this file might want to use or refer to. By placing these @b[requires] calls here, we make it possible for the packages being loaded to refer to external symbols in this package. Any number of @b[use-package] and @b[import] calls, to make it the symbols from other packages available in this package. Finally, the definitions making up the contents of this package/module. @End[enumerate] Now, suppose that the @b[phlogiston] and @b[alchemy] packages are single-file, single-package modules as described above. Phlogiston wants to use the alchemy package, and alchemy wants to use several external symbols from phlogiston. The following definitions allow the user to supply @b[require] statement for either of these modules, or for both of them in either order. The file "alchemy.lisp": @begin [lisp] ;;; Alchemy functions, written and maintained by Merlin, Inc. ;;; Both the module and the package are named "alchemy". (provide "alchemy") (in-package "alchemy") ;;; Nothing to shadow. ;;; Here is our external interface. (export '(lead-to-gold gold-to-lead antimony-to-zinc elixir-of-life)) ;;; This file uses some functions from the phlogiston package/module. (require "phlogiston") ;;; We use this a lot, so import it. It's external in phlogiston package. (import '(phlogiston:make-fire-bottle)) ;;; Now for the real contents of this file. (defun lead-to-gold (x) "Takes a quantity of lead, returns gold." (when (> (phlogiston:heat-flow x) 3) ; Using a qualified symbol. (make-fire-bottle x)) ; Using an imported symbol. (gild x)) ;;; And so on ... @end [lisp] The file "phlogiston.lisp": @begin [lisp] ;;; Phlogiston functions, written and maintained by Thermofluidics, Ltd. ;;; Both the module and the package are named "phlogiston". (provide "phlogiston") (in-package "phlogiston") ;;; Nothing to shadow. ;;; Here is our external interface. (export '(heat-flow cold-flow mix-fluids separate-fluids burn make-fire-bottle)) ;;; This file uses some functions from the alchemy package/module. (require "alchemy") ;;; We use alchemy functions a lot, so use the package. (use-package "alchemy") ;;; The real contents of this package/module. (defun heat-flow (amount x y) "Make some amount of heat flow from x to y." (when feeling-weak (quaff (elixir-of-life))) ; No qualifier needed. (push-heat amount x y)) ;;; And so on ... @end [lisp] For very large packages whose contents are spread over several files (the @b[lisp] package is an example), it is recommended that the author create the package and declare all of the shadows and external symbols in a separate file, so that this can be loaded before anything which might use symbols from this package.  Received: from CMU-CS-C by SU-AI with TCP/SMTP; 18 May 83 21:22:49 PDT Received: ID ; 19 May 83 00:23:20 EDT Date: Thu, 19 May 1983 00:23 EDT From: Scott E. Fahlman To: common-lisp@su-ai Cc: les@CMU-CS-C Subject: Maphash One of my hackers has just noticed that we still map over the elements of a hashtable with MAPHASH, while we have changed most similar forms to DO-mumble, as in DO-SYMBOLS, etc. Since MAPHASH returns NIL, there is no elegant way to turn the contents of a hashtable into a list. What would people think of flushing MAPHASH and replacing it with DO-HASH ? -- Scott  Received: from MIT-MC by SU-AI with TCP/SMTP; 18 May 83 13:44:12 PDT Received: from SCRC-EUPHRATES by SCRC-TENEX with CHAOS; Wed 18-May-83 16:47:27-EDT Date: Wednesday, 18 May 1983, 16:45-EDT From: David A. Moon Subject: writeup on name conflicts To: Common-Lisp@su-ai @section[Name Conflicts] A fundamental invariant of the package system is that within one package any particular name can only refer to one symbol. A @i[name conflict] is said to occur when there is more than one candidate symbol and it is not obvious which one to choose. If the system does not always choose the same way, the read-read consistency rule would be violated. For example, some programs or data might have been read in under a certain mapping of the name to a symbol. If the mapping changes to a different symbol, then additional programs or data are read, the two programs will not access the same symbol even though they use the same name. Even if the system did always choose the same way, a name conflict is likely to result in a different mapping from names to symbols than was expected by the user, causing programs to execute incorrectly. Therefore, any time a name conflict occurs, an error is signalled. The user may continue from the error and tell the package system how to resolve the conflict. Note that if the same symbol is accessible to a package through more than one path, for instance as an external of more than one package, or both through inheritance and through direct presence in the package, there is no name conflict. Name conflicts only occur between distinct symbols with the same name. The creator of a package can tell the system in advance how to resolve a name conflict through the use of @i[shadowing]. Every package has a list of shadowing symbols. A shadowing symbol takes precedence over any other symbol of the same name that would otherwise be accessible to the package. A name conflict involving a shadowing symbol is always resolved in favor of the shadowing symbol, without signalling an error (except for one exception involving @b[import] described below). The functions @b[shadow] and @b[shadowing-import] may be used to declare shadowing symbols. Name conflicts are detected when they become possible, i.e. when the package structure is altered. There is no need to check for name conflicts during every name lookup. The functions @b[use-package], @b[import], and @b[export] check for name conflicts. @b[use-package] makes the external symbols of the package being used accessible to the using package; each of these symbols is checked for name conflicts with the symbols already accessible. @b[import] adds a single symbol to the internals of a package, checking for a name conflict with an existing symbol either present in the package or accessible to it. @b[import] signals an error even if there is a name conflict with a shadowing symbol, because two explicit directives from the user are inconsistent. @b[export] makes a single symbol accessible to all the packages that use the package from which the symbol is exported. All of these packages are checked for name conflicts: @b[(export @i[s] @i[p])] does @b[(find-symbol (symbol-name @i[s]) @i[q])] for each package @i[q] in @b[(package-used-by-list @i[p])]. Note that in the usual case of an @b[export] during the initial definition of a package, the @b[package-used-by-list] will be @b[nil] and the name conflict checking will take no time. @b[intern] does not need to do any name-conflict checking, because it never creates a new symbol if there is already an accessible symbol with the name given. @b[shadow] and @b[shadowing-import] never signal a name-conflict error, because by calling these functions the user has specified how any possible conflict is to be resolved. @b[shadow] does name-conflict checking to the extent that it checks whether a distinct existing symbol with the specified name is accessible, and if so whether it is directly present in the package or inherited; in the latter case a new symbol is created to shadow it. @b[shadowing-import] does name-conflict checking to the extent that it checks whether a distinct existing symbol with the same name is accessible; if so it is shadowed by the new symbol, which implies that it must be @b[unintern]ed if it was directly present in the package. @b[unuse-package], @b[unexport], and @b[unintern] (when the symbol being @b[unintern]ed is not a shadowing symbol) do not need to do any name-conflict checking, because they only remove symbols from a package; they do not make any new symbols accessible. @b[unintern] of a shadowing symbol can uncover a name conflict that had previously been resolved by the shadowing. If package A uses packages B and C, A contains a shadowing symbol @b[x], and B and C each contain external symbols named @b[x], then @b[unintern]ing @b[x] from A will reveal a name conflict between @b[b:x] and @b[c:x] if those two symbols are distinct. In this case @b[unintern] will signal an error. Aborting from a name-conflict error leaves the original symbol accessible. Package functions always signal name-conflict errors before making any change to the package structure. Note: when multiple changes are to be made, for example when @b[export] is given a list of symbols, it is legal for each change to be processed separately, so that aborting from a name conflict caused by the second symbol in the list will not unexport the first symbol in the list. However, aborting from a name conflict error caused by @b[export] of a single symbol will not leave that symbol accessible to some packages and inaccessible to others; @b[export] appears as an atomic operation. Continuing from a name-conflict error should offer the user a chance to resolve the name conflict in favor of either of the candidates. The package structure should be altered to reflect the resolution of the name conflict, via @b[shadowing-import], @b[unintern], or @b[unexport]. A name conflict in @b[use-package] between a symbol directly present in the using package and an external symbol of the used package may be resolved in favor of the first symbol by making it a shadowing symbol, or in favor of the second symbol by @b[unintern]ing the first symbol from the using package. The latter resolution is dangerous if the symbol to be @b[unintern]ed is an external symbol of the using package since it will cease to be an external symbol. A name conflict in @b[use-package] between two external symbols inherited by the using package from other packages may be resolved in favor of either symbol by importing it into the using package and making it a shadowing symbol. A name conflict in @b[export] between the symbol being exported and a symbol already present in a package that would inherit the newly-exported symbol may be resolved in favor of the exported symbol by @b[unintern]ing the other one, or in favor of the already-present symbol by making it a shadowing symbol. A name conflict in @b[export] or @b[unintern] due to a package inheriting two distinct symbols with the same name from two other packages may be resolved in favor of either symbol by importing it into the using package and making it a shadowing symbol, just as with @b[use-package]. A name conflict in @b[import] between the symbol being imported and a symbol inherited from some other package may be resolved in favor of the symbol being imported by making it a shadowing symbol, or in favor of the symbol already accessible by not doing the @b[import]. A name conflict in @b[import] with a symbol already present in the package may be resolved by @b[unintern]ing that symbol, or by not doing the @b[import]. Good user-interface style dictates that @b[use-package] and @b[export], which can cause many name conflicts simultaneously, first check for all of the name conflicts before presenting any of them to the user. The user may then choose to resolve all of them wholesale, or to resolve each of them individually, requiring a lot of interaction but permitting different conflicts to be resolved different ways. Some implementations will have other ways of resolving name conflicts to offer. For instance, if the symbols that conflict are not being used as objects, but only as names for functions, it may be possible to "merge" the two symbols by putting the function definition onto both symbols. References to either symbol for purposes of calling a function would be equivalent. A similar merging operation can be done for variable values and for things stored on the property list. On the Lisp machine one can also @i[forward] the value, function, and property cells so that future changes to either symbol will propagate to the other one. Some other implementations are able to do this with value cells, but not with property lists. Only the user can know whether this way of resolving a name conflict is adequate, i.e. the fact that there are two non-@b[eq] symbols with the same name doesn't affect the correct operation of his program. The value of offering symbol-merging as a way of resolving name conflicts is that it can avoid the need to throw away the whole Lisp world and start over, after correcting the package-definition forms that caused the error.  Received: from MIT-MC by SU-AI with TCP/SMTP; 18 May 83 10:29:55 PDT Date: Wednesday, 18 May 1983 13:30-EDT From: MOON at SCRC-TENEX To: Common-Lisp at su-ai Subject:revised BREAK writeup BREAK &optional format-string &rest args Print the message and go directly into the debugger, without allowing any possibility of interception by programmed error-handling facilities. There aren't any error-handling facilities in Common Lisp, but there might be in particular implementations, and there will be in Common Lisp in the future. When continued, BREAK returns NIL. It is permissible to call BREAK with no arguments; it will supply some default message. BREAK is presumed to be used as a way of inserting temporary debugging "breakpoints" in a program, not as a way of signalling errors. Thus continuing from a BREAK would never do anything special, and it does not take the second FORMAT control-string argument that CERROR takes. This and the lack of any possibility of interception by programmed error-handling are the only program-visible differences between BREAK and CERROR. The interactive debugger may choose to display them differently, for instance a CERROR's message might be prefixed with "Error:" and a BREAK's message prefixed with "Break:". This depends on the user-interface style of the particular implementation. A particular implementation may choose, according to its own style and needs, to go into a different debugger when BREAK is called than when an error occurs. For example, it might go into a plain read-eval-print loop identical to the top-level one except for the provision of a "continue" command that causes BREAK to return NIL. Compatibility note: Maclisp's BREAK takes two optional arguments. The first would be a string if Maclisp had strings. The second is a boolean value specifying whether BREAK should break or return immediately. In Common Lisp one makes a BREAK conditional by putting it inside a conditional form such as WHEN or UNLESS.  Received: from MIT-MC by SU-AI with TCP/SMTP; 18 May 83 00:03:46 PDT Received: from SCRC-EUPHRATES by SCRC-TENEX with CHAOS; Wed 18-May-83 03:04:58-EDT Date: Wednesday, 18 May 1983, 03:03-EDT From: David A. Moon Subject: Conservation of nits To: common-lisp@su-ai In-reply-to: The message of 18 May 83 00:08-EDT from Scott E. Fahlman Date: Wed, 18 May 1983 00:08 EDT From: Scott E. Fahlman From: Moon I hope you realize that IN-PACKAGE is incompatible with incremental re-compilation of portions of files (after editing them), a Lisp machine productivity feature that users are uniformly enthusiastic about. I would think that this would be something that Spice Lisp would definitely want, too, once you have a compiler that runs natively in your Lisp. I'm not sure yet whether we will strongly discourage people from using IN-PACKAGE, or whether we will provide a kludgey way for the editor to know that it must parse the beginning of the file looking for IN-PACKAGE in order to determine in what package to read forms from the file. I don't completely understand this. I looked for incremental compilation in the Confederate edition of the Chine Nual and couldn't find anything about it. What exactly do you folks do? I would have assumed that if you do incremental re-compilation on an ASCII file, you take some notice of the -*-Package:foo-*- business, no? IN-PACKAGE would have the same set of problems as that. I would hate to have you tell users not to use IN-PACKAGE. Do you have a specific proposal? The Chine Nual doesn't document the editor. Incremental compilation = designate some portion of the buffer in the editor (in the simplest case it's the top-level form surrounding the editor cursor) and compile it putting the results into the current environment. You can also just read/eval/print it. To do this the editor needs to know the base, readtable, and package required to read the text (it also needs to know what language it's in, but that's a different story). The way we tell the editor this information is to put it all in the file attribute line, which is the first non-blank line in the file and contains the characters "-*-". The important point about this is not the particular representation of this information, but the fact that it is the same for everything in the file. When recompiling something halfway down the file, you don't need to read everything from the beginning of the file up to there to find out what package (etc.) is needed to read the one form being compiled. The only thing that is important about the particular representation of the information is that you can find it, or determine that it is absent, without reading through the whole file, and in fact there is a general facility for dealing with these file attribute lines, used by the editor, the compiler, the loader, and any other programs that care. Doing it with Lisp forms that you evaluate, such as IN-PACKAGE, works for the compiler and the loader, but for other programs (the editor is an example) whose business is not to evaluate Lisp forms it doesn't fit in so smoothly. So one issue is allowing the package (etc.) to change in the middle of a file. This seems of little worth to me, so I doubt we will make any effort to support incremental compilation of files that do this. We will of course support non-incremental compilation and loading of such files if Common Lisp allows them. The other issue is whether we want (not in Common Lisp, but in our own Lisp machine user interface) to introduce a new syntax for saying what package (etc.) a file is in, when we already have one. Probably we will just say that you put the information in both places if you want your file to be incrementally compilable and you want it to work in other Common Lisp implementations. My concern is not that we (Symbolics) can't accept IN-PACKAGE, I was more trying to point out that you (Spice) might be backing yourselves into a corner without realizing it, by adopting a style of programming that is inimicable to incremental compilation. The writeup doesn't say why certain functions should be used at top level. One reason is that if changes are being made to *package* with the expectation that they will affect how forms later in the file are read, then those changes had better happen at compile time before further forms are read from the file. The other reason is that in many implementations the output from the compiler is designed so that the loader can look symbols up once, rather having to call INTERN every time a symbol is referenced. This means that the compiler must know when the package state is changing (i.e. the read-read consistency rule doesn't apply). It would probably be best to include an Implementation Note pointing out the restrictions this places on the compiler/loader implementation: the compiler may not read the whole file before compiling any of it (as the Lisp machine used to do in order to improve virtual memory locality); the loader may not do all its INTERNs before evaling any of the top-level forms (as Multics Maclisp does). The writeup should say explicitly which functions get an automatic (eval-when (compile) ...) wrapped around them when they are seen at top level by the compiler. The example at the end is assuming this implicitly. Well, both of the above comments assume that the compiler represents symbols and their packages internally just by reading them into the right package. Certainly the compiler has to keep track of what packages the various symbols want to end up in, but this might be done by some more complex mechanism if the compiler wanted to keep its guts somewhat separate from the stuff it is reading. I'll see if I can say something sufficient about this without over-specifying the mechanism. I don't think macros can reasonably run at compile time if the compiler doesn't represent symbols in the code to be compiled as symbols. Your last sentence above sounds like the right thing. I can find no justification in the previous discussion for the suddenly introduced restriction that EXPORT and UNEXPORT are not allowed in packages that have been USE-PACKAGE'd, unless this is a Solomonian attempt to resolve the copy vs reference issue. If this is trying to deal with the case of exporting a symbol too late, after someone who thought they were getting it via USE-PACKAGE made an internal symbol instead, the naming conflict checks I suggested earlier detect this and are able to tell you exactly what happened. If this is trying to deal with the case where exporting a new symbol, that people using USE-PACKAGE didn't expect to see exported, causes what was expected to be two distinct symbols, internal in different packages, to become one (external) symbol, it doesn't deal with it. That can just as well happen if all the EXPORTs are done before all the USE-PACKAGEs, and is basically indetectable. This screw case is really a lot like someone changing what a function does without telling everyone who uses it. I was just trying to do something simple and safe that might end the debate. I figured that I could get this "package locking" rule right on the first try, and that some tricky rule to signal an error only in the case of a conflict with some symbol in the inferior package that is not on that package's shadow list would lead to N more rounds of revision. You seem to understand this better than I do. How about if you propose the exact wording for the error-checking rule you would like to see here (both for EXPORT and for UNEXPORT)? That would be the quickest way to converge, I think. Okay. But not in this message, I'm too tired to write coherently. Tomorrow. Just out of curiosity, why in the world would anyone compile an INIT file? To make it load faster, or to make the dozens of functions defined in it run faster. You should see the init files of some of our people!  Received: from MIT-MC by SU-AI with TCP/SMTP; 17 May 83 23:20:16 PDT Received: from SCRC-EUPHRATES by SCRC-TENEX with CHAOS; Wed 18-May-83 02:11:07-EDT Date: Wednesday, 18 May 1983, 02:09-EDT From: David A. Moon Subject: Break To: common-lisp@su-ai In-reply-to: The message of 17 May 83 22:16-EDT from Scott E. Fahlman Date: Tue, 17 May 1983 22:16 EDT From: Scott E. Fahlman How about if we just say that BREAK enters the debugger or a recursive read/eval/print loop at the implementation's discretion? Okay  Received: from CMU-CS-C by SU-AI with TCP/SMTP; 17 May 83 21:09:04 PDT Received: ID ; 18 May 83 00:08:23 EDT Date: Wed, 18 May 1983 00:08 EDT From: Scott E. Fahlman To: David A. Moon Cc: common-lisp@su-ai Subject: Conservation of nits In-reply-to: Msg of 16 May 1983 20:51-EDT from David A. Moon Fahlman's Law of Package Systems: Nits can neither be created nor destroyed. It is occasionally possible to smash big ones into a whole lot of little ones. I'd suggest that duplicate package name be an error for MAKE-PACKAGE, a primitive, but not for IN-PACKAGE, since you'd like to be able to load a program twice (presumably after changing it to fix something). OK. That's the way I had it originally, before our go-round about IN-PACKAGE augmenting the externals of the package in question. I hope you realize that IN-PACKAGE is incompatible with incremental re-compilation of portions of files (after editing them), a Lisp machine productivity feature that users are uniformly enthusiastic about. I would think that this would be something that Spice Lisp would definitely want, too, once you have a compiler that runs natively in your Lisp. I'm not sure yet whether we will strongly discourage people from using IN-PACKAGE, or whether we will provide a kludgey way for the editor to know that it must parse the beginning of the file looking for IN-PACKAGE in order to determine in what package to read forms from the file. I don't completely understand this. I looked for incremental compilation in the Confederate edition of the Chine Nual and couldn't find anything about it. What exactly do you folks do? I would have assumed that if you do incremental re-compilation on an ASCII file, you take some notice of the -*-Package:foo-*- business, no? IN-PACKAGE would have the same set of problems as that. I would hate to have you tell users not to use IN-PACKAGE. Do you have a specific proposal? Say somewhere that the accessor function for the home package of a symbol is SYMBOL-PACKAGE. OK It isn't clear from the description whether SHADOWING-IMPORT will call UNINTERN if the symbol being shadowed is directly in the package (rather than inherited with USE-PACKAGE). Surely this must be what was intended. Also SHADOWING-IMPORT should be in the list of functions that should be used at top level. OK The writeup doesn't say why certain functions should be used at top level. One reason is that if changes are being made to *package* with the expectation that they will affect how forms later in the file are read, then those changes had better happen at compile time before further forms are read from the file. The other reason is that in many implementations the output from the compiler is designed so that the loader can look symbols up once, rather having to call INTERN every time a symbol is referenced. This means that the compiler must know when the package state is changing (i.e. the read-read consistency rule doesn't apply). It would probably be best to include an Implementation Note pointing out the restrictions this places on the compiler/loader implementation: the compiler may not read the whole file before compiling any of it (as the Lisp machine used to do in order to improve virtual memory locality); the loader may not do all its INTERNs before evaling any of the top-level forms (as Multics Maclisp does). The writeup should say explicitly which functions get an automatic (eval-when (compile) ...) wrapped around them when they are seen at top level by the compiler. The example at the end is assuming this implicitly. Well, both of the above comments assume that the compiler represents symbols and their packages internally just by reading them into the right package. Certainly the compiler has to keep track of what packages the various symbols want to end up in, but this might be done by some more complex mechanism if the compiler wanted to keep its guts somewhat separate from the stuff it is reading. I'll see if I can say something sufficient about this without over-specifying the mechanism. I can find no justification in the previous discussion for the suddenly introduced restriction that EXPORT and UNEXPORT are not allowed in packages that have been USE-PACKAGE'd, unless this is a Solomonian attempt to resolve the copy vs reference issue. If this is trying to deal with the case of exporting a symbol too late, after someone who thought they were getting it via USE-PACKAGE made an internal symbol instead, the naming conflict checks I suggested earlier detect this and are able to tell you exactly what happened. If this is trying to deal with the case where exporting a new symbol, that people using USE-PACKAGE didn't expect to see exported, causes what was expected to be two distinct symbols, internal in different packages, to become one (external) symbol, it doesn't deal with it. That can just as well happen if all the EXPORTs are done before all the USE-PACKAGEs, and is basically indetectable. This screw case is really a lot like someone changing what a function does without telling everyone who uses it. I was just trying to do something simple and safe that might end the debate. I figured that I could get this "package locking" rule right on the first try, and that some tricky rule to signal an error only in the case of a conflict with some symbol in the inferior package that is not on that package's shadow list would lead to N more rounds of revision. You seem to understand this better than I do. How about if you propose the exact wording for the error-checking rule you would like to see here (both for EXPORT and for UNEXPORT)? That would be the quickest way to converge, I think. The writeup says that SHADOW and SHADOWING-IMPORT "can produce confusing situations that violate the consistency rules." This is misleading. They are a change of package state, just like setting *PACKAGE*, so there isn't consistency between the world before calling SHADOW and the world after calling it. But the world after calling SHADOW is consistent with itself. OK, good point, sloppy wording. The &optional package argument to USE-PACKAGE and UNUSE-PACKAGE is missing. OOPS. Why does it say init files are not usually compiled? This is a non-sequitur where it appears, and also is untrue in some implementations, ours for instance. Weinreb and Greenberg sent me a message pointing out that Newton's init file would not work if compiled, and that it needed some (eval-when (compile ..)) stuff added. Since this was supposed to the simple part of the example (the sort of interaction that naive users might have with the package system), I didn't want to hair it up that way, so I just mentioned that you don't want to compile these things. Actually, I guess with the recent changes, and if we put in the comment about implicit EVAL-WHENs, we don't need to worry about this. Just out of curiosity, why in the world would anyone compile an INIT file? Why do two of the modules in Newton's init file have ".lsp" in their names? OOPS, again. I'll try to get this fixed up tomorrow. -- Scott  Received: from CMU-CS-C by SU-AI with TCP/SMTP; 17 May 83 19:17:14 PDT Received: ID ; 17 May 83 22:16:19 EDT Date: Tue, 17 May 1983 22:16 EDT From: Scott E. Fahlman To: common-lisp@su-ai Subject: Break How about if we just say that BREAK enters the debugger or a recursive read/eval/print loop at the implementation's discretion? Attempting to nail down exactly what happens here is probably a bad idea, given the variety of user-interface styles that we will want to support. I find it pretty hard to believe that people are worried about the one keystroke that it ought to take to get from one kind of loop to the other in any event. Can anyone NOT live with this? -- Scott  Received: from SUMEX-AIM by SU-AI with PUP; 17-May-83 08:18 PDT Received: from UTAH-20 by SUMEX-AIM.ARPA with TCP; Tue 17 May 83 05:37:40-PDT Date: 17 May 1983 0633-MDT From: Martin.Griss Subject: Debug vs Break To: common-lisp@SU-AI cc: griss@UTAH-20 I'm not entirely familiar with LispM debug vs Break. In PSL, the Break loop IS a read-eval-print loop; it has a distinguished prompt, and a few characters/ids are enabled as aliases for functions. E.g., Q -> (BREAK-QUIT) T -> (BREAK-BACK-TRACE) C -> (BREAK-CONTINUE) R -> (BREAK-RETRY) etc. We could envision that the single stepper is also present, and off, but ready to be invoked in a retry mode, by say S, or some such. In an environment that we are developing using a multi-window EMACS like editor (NMODE), we have experimented with a "pop-up" window/buffer in which the single chars above are present and ready to be pointed at and executed (a "menu"). Normal editing and execution of s-expressions can proceed. The Break/Debug commands can also be bound to appropriate CNTRL-keys (e.g. LISP-Q, LISP-A), via an appropriate dispatch table for BREAK-MODE, a modified LISP-MODE. Various combinations of these techniques can gracefully merge a "read-eval-print" break loop, with a Debug loop. Martin -------  Received: from MIT-MC by SU-AI with TCP/SMTP; 16 May 83 21:03:47 PDT Received: from SCRC-EUPHRATES by SCRC-TENEX with CHAOS; Mon 16-May-83 21:08:25-EDT Date: Monday, 16 May 1983, 21:06-EDT From: David A. Moon Subject: error.proposal.9 To: Kent M. Pitman Cc: Common-Lisp@SU-AI In-reply-to: The message of 16 May 83 20:43-EDT from Kent M. Pitman Date: 16 May 1983 20:43 EDT From: Kent M. Pitman How literally do you mean that BREAK should enter the debugger? Quite literally. On Maclisp and even now on the LispM, when you type Break, you are at a place where you type s-expressions, not commands. The BREAK function and the Suspend key (labelled Break on old keyboards) are not the same. Indeed I propose to change the BREAK function incompatibly with Maclisp and Zetalisp. The name of the function would have been changed, too, but no one liked any alternative names. If other implementations want BREAK to go into a read-eval-print loop, I am amenable to weakening the wording in the manual. But I think it would be much better to provide a function specifically for doing a read-eval-print loop. BREAK isn't really such a good name for that function, since if BREAK means anything at all, it means "breakpoint" which is historically associated with debuggers.  Received: from MIT-MC by SU-AI with TCP/SMTP; 16 May 83 20:48:04 PDT Received: from SCRC-EUPHRATES by SCRC-TENEX with CHAOS; Mon 16-May-83 20:53:24-EDT Date: Monday, 16 May 1983, 20:51-EDT From: David A. Moon Subject: packages, yet again To: common-lisp@su-ai In-reply-to: The message of 16 May 83 00:48-EDT from Scott E. Fahlman I am quite happy with this version of the package proposal. There are still some minor nits to pick: It doesn't say explicitly what happens if MAKE-PACKAGE is done twice with the same name. Is this an error? Is it an error for IN-PACKAGE, too? Is there any attempt to detect two people independently choosing the same package name, or is that expliclitly considered impractical to detect? I'd suggest that duplicate package name be an error for MAKE-PACKAGE, a primitive, but not for IN-PACKAGE, since you'd like to be able to load a program twice (presumably after changing it to fix something). I hope you realize that IN-PACKAGE is incompatible with incremental re-compilation of portions of files (after editing them), a Lisp machine productivity feature that users are uniformly enthusiastic about. I would think that this would be something that Spice Lisp would definitely want, too, once you have a compiler that runs natively in your Lisp. I'm not sure yet whether we will strongly discourage people from using IN-PACKAGE, or whether we will provide a kludgey way for the editor to know that it must parse the beginning of the file looking for IN-PACKAGE in order to determine in what package to read forms from the file. Say somewhere that the accessor function for the home package of a symbol is SYMBOL-PACKAGE. It isn't clear from the description whether SHADOWING-IMPORT will call UNINTERN if the symbol being shadowed is directly in the package (rather than inherited with USE-PACKAGE). Surely this must be what was intended. Also SHADOWING-IMPORT should be in the list of functions that should be used at top level. The writeup doesn't say why certain functions should be used at top level. One reason is that if changes are being made to *package* with the expectation that they will affect how forms later in the file are read, then those changes had better happen at compile time before further forms are read from the file. The other reason is that in many implementations the output from the compiler is designed so that the loader can look symbols up once, rather having to call INTERN every time a symbol is referenced. This means that the compiler must know when the package state is changing (i.e. the read-read consistency rule doesn't apply). It would probably be best to include an Implementation Note pointing out the restrictions this places on the compiler/loader implementation: the compiler may not read the whole file before compiling any of it (as the Lisp machine used to do in order to improve virtual memory locality); the loader may not do all its INTERNs before evaling any of the top-level forms (as Multics Maclisp does). The writeup should say explicitly which functions get an automatic (eval-when (compile) ...) wrapped around them when they are seen at top level by the compiler. The example at the end is assuming this implicitly. I can find no justification in the previous discussion for the suddenly introduced restriction that EXPORT and UNEXPORT are not allowed in packages that have been USE-PACKAGE'd, unless this is a Solomonian attempt to resolve the copy vs reference issue. If this is trying to deal with the case of exporting a symbol too late, after someone who thought they were getting it via USE-PACKAGE made an internal symbol instead, the naming conflict checks I suggested earlier detect this and are able to tell you exactly what happened. If this is trying to deal with the case where exporting a new symbol, that people using USE-PACKAGE didn't expect to see exported, causes what was expected to be two distinct symbols, internal in different packages, to become one (external) symbol, it doesn't deal with it. That can just as well happen if all the EXPORTs are done before all the USE-PACKAGEs, and is basically indetectable. This screw case is really a lot like someone changing what a function does without telling everyone who uses it. The writeup says that SHADOW and SHADOWING-IMPORT "can produce confusing situations that violate the consistency rules." This is misleading. They are a change of package state, just like setting *PACKAGE*, so there isn't consistency between the world before calling SHADOW and the world after calling it. But the world after calling SHADOW is consistent with itself. The &optional package argument to USE-PACKAGE and UNUSE-PACKAGE is missing. Why does it say init files are not usually compiled? This is a non-sequitur where it appears, and also is untrue in some implementations, ours for instance. Why do two of the modules in Newton's init file have ".lsp" in their names?  Received: from CMU-CS-C by SU-AI with TCP/SMTP; 16 May 83 20:39:59 PDT Received: ID ; 16 May 83 23:39:40 EDT Date: Mon, 16 May 1983 23:39 EDT From: Scott E. Fahlman To: David A. Moon Cc: Common-Lisp@SU-AI Subject: args to error, cerror, & warn In-reply-to: Msg of 16 May 1983 20:59-EDT from David A. Moon I'm sort of on the fence about whether a place-holder for the condition name would be a good idea. However, given Moon's strong feelings on the subject, I think we would prefer an optional first argument to three new functions. -- Scott  Received: from MIT-MC by SU-AI with TCP/SMTP; 16 May 83 20:32:46 PDT Received: from SCRC-EUPHRATES by SCRC-TENEX with CHAOS; Mon 16-May-83 21:01:50-EDT Date: Monday, 16 May 1983, 20:59-EDT From: David A. Moon Subject: args to error, cerror, & warn To: Common-Lisp@SU-AI In-reply-to: The message of 16 May 83 19:46-EDT from Glenn S. Burke Date: 16 May 1983 19:46 EDT From: Glenn S. Burke Subject: args to error, cerror, & warn I think we should require a first argument of NIL there now for a place-holder since we are not going to define the extension yet. Optional first arguments are a pain for both implementation and documentation. I don't think it would be a good idea to require that NIL be written when the first argument is omitted, since all naive users will be omitting the first argument. The implementation pain is negligible since there is no reason for these functions to be fast (anyway, they call FORMAT!). The documentation pain is real, but I think it is outweighed by the convenience of having conditions be invisible to naive users. If the concensus of the committee is that we can't have optional first arguments, then I will insist that the future extension to conditions be made by introducing three new functions so that the basic functions don't have to be called with an extraneous argument.  Received: from MIT-MC by SU-AI with TCP/SMTP; 16 May 83 20:15:59 PDT Date: 16 May 1983 20:43 EDT From: Kent M. Pitman Subject: error.proposal.9 To: Moon @ SCRC-TENEX cc: Common-Lisp @ SU-AI How literally do you mean that BREAK should enter the debugger? On Maclisp and even now on the LispM, when you type Break, you are at a place where you type s-expressions, not commands. On the LispM, you have a wide enough character set to be able to merge the break facility with the debugger and still win, but in ASCII land, this will be a lot tougher. I, for one, will be unhappy with BREAK putting me somewhere where the first thing I have to type is a command to get me to a vanilla read eval print loop style debugging environment. I suggest that the wording should be weakened so that BREAK is constrained only to pause and allow you access to debugging state of some sort, but it should be up to the implementation whether a debugger or read eval print loop is entered. I think this is in the domain of error handling and is something and should at this point be left to individual implementations. Alternatively, I would not mind (I would even prefer) two similar commands, like DBG and BREAK on the Lisp Machine, so I could call the one which was appropriate for the kind of debugging I am doing. I will almost always know in advance which of these debugging environments I want. If they happen to be different entry points to the same program (as I suspect will eventually be the case with the LispM), that's fine. It's just a question of intent. The former suggestion is probably most practical at this point. Other than that, I think error.proposal.9 looks fine. --kmp  Received: from USC-ECL by SU-AI with TCP/SMTP; 16 May 83 16:47:34 PDT Received: from MIT-ML by USC-ECL; Mon 16 May 83 16:45:34-PDT Date: 16 May 1983 19:46 EDT From: Glenn S. Burke Subject: args to error, cerror, & warn To: Common-Lisp @ SU-AI I think we should require a first argument of NIL there now for a place-holder since we are not going to define the extension yet. Optional first arguments are a pain for both implementation and documentation. (Many of us have been doing this for years with FERROR in maclisp anyway, where it was never really well-defined what a non-null first argument meant.)  Received: from MIT-MC by SU-AI with TCP/SMTP; 16 May 83 16:01:25 PDT Received: from SCRC-EUPHRATES by SCRC-TENEX with CHAOS; Mon 16-May-83 19:04:59-EDT Date: Monday, 16 May 1983, 19:02-EDT From: David A. Moon Subject: error.proposal.9 To: Common-Lisp@su-ai This is probably the final version of my error proposal. Unless there are objections it is ready to convert to Scribe, render style-compatible with the Common Lisp manual, and adopt. Our experience is that error-handling of adequate flexibility and power requires a complex error classification system, using flavors or the equivalent. Not everyone in the Common Lisp community agrees, and our implementation of these ideas has only been in existence for half a year, not long enough to consider it mature and stable. Chapter 23 of the laser manual is mainly a rehash of the old Lisp machine condition system (documented in The Lisp Machine Manual), which was generally found to be totally inadequate. Therefore we propose that for the present the language only standardize on ways to SIGNAL errors, not on ways to HANDLE errors. Of course a complete language requires error-handling mechanisms, but many useful portable programs do not require them. Thus it makes sense to defer standardizing this until we understand it better. The rest of this proposal replaces everything in chapter 23 of the laser manual. Most uppercase text should be read as boldface. Italics are not indicated. - Handling errors When an error is signalled, either by calling one of the functions documented in this section or directly by the Lisp system, it is handled in an implementation-dependent way. The general assumption is that each implementation will provide an interactive debugger that prints the error message, along with suitable contextual information such as what function detected the error. The user may interact with the debugger to examine or modify the state of the program in various ways, including abandoning the current computation ("aborting to top level") and continuing from the error. What "continuing" means depends on how the error is signalled; details are given with each error signalling function. [The Lisp machine calls continuing "proceeding".] - General error signalling functions ERROR format-string &rest args Signal an error, with the message constructed by applying FORMAT to the arguments. It is impossible to continue from this kind of error; thus the ERROR function will never return (barring someone meddling with it by redefining it or by using an interactive debugger to force it to return). The error message should not contain a carriage return at the beginning or end, and should not contain any sort of herald indicating that it is an error. The system will take care of these according to whatever its preferred style may be. Conventionally error messages are complete English sentences, ending with a period. Carriage returns in the middle of long messages are okay. There should be no indentation after a carriage return in the middle of an error message. If the debugger in a particular implementation displays error messages indented (e.g. indented by 7 spaces because they are prefixed by "Error: ") then it should take care of inserting the appropriate amount of indentation into a multi-line error message. Similarly, a debugger that prefixes error messages with semicolons should take care of inserting a semicolon at the beginning of each line in a multi-line error message. Note that even within a single implementation, there may be more than one program that presents error messages to the user, and they may use different styles, so the caller of ERROR cannot provide for this in advance. The debugger printout in the following example is not to be taken as a Common Lisp requirement; it is only an example of what an implementation might print when ERROR is called. Example: (ERROR "The command ~S is unrecognized." CMD) Error: The command EMERGECNY-SHUTDOWN is unrecognized. Error signalled by function OPERATOR-COMMAND-EXECUTE. > [The Lisp machine calls this FERROR]. CERROR error-format-string continue-format-string &rest args Signal an error, with the message constructed by applying FORMAT to error-format-string and args. Continuing from such an error will cause CERROR to return NIL. Use CERROR rather than ERROR to signal errors for which you have written recovery code. The name stands for "continuable error," which is too verbose to use for such a common function. A message describing what will happen if the error is continued will be constructed by applying FORMAT to continue-format-string and args. This message should be a one-line description of the effect of continuing. It should be phrased as an English sentence in the imperative mood, just like the documentation of a command or a function. Typically it would be used by an interactive debugger as the documentation of its "continue" command. The message supplied with CERROR should not include any statement of how the "continue" command is given, since this may be different for each debugger. The debugger will insert this into the message, according to its own particular user-interface style. Examples: (UNLESS (= (LIST-LENGTH FORM) 3) (CERROR "Wrong number of arguments in ~S" (IF (< (LIST-LENGTH FORM) 3) "Assume 0 for missing args." "Ignore extra args.") FORM) (SETQ FORM (APPEND FORM '(0 0)))) Error: Wrong number of arguments in (F X) Error signalled by function EXAMPLE. If continued: Assume 0 for missing args. > (DO () ((KNOWN-WORDP X) X) (CERROR "~S is unknown, probably misspelled." "Replace ~S and try again." X) (FORMAT T "~&New word: ") (SETQ X (READ))) ;This example could be done more briefly with ASSERT or ;even with CHECK-TYPE (and SATISFIES). In complex cases where the error message uses some of the args and the continue message uses others of the args, it may be necessary to use the ~G or ~* operators of FORMAT to skip over unwanted arguments in one of the FORMAT control strings. [The Lisp machine calls this FSIGNAL, except that it returns :NO-ACTION rather than NIL and it fails to distinguish between the error message and the continue message.] WARN format-string &rest args Print an error message constructed by applying FORMAT to the arguments, but don't go into the debugger. WARN returns NIL. This would just be FORMAT with output directed to *ERROR-OUTPUT*, except that WARN takes care of advancing to a fresh line before and after the error message and may do other implementation-dependent things (for example, in Symbolics Common Lisp, WARN messages printed during a compilation are associated with the function being compiled and saved for later perusal by editor tools. Furthermore WARN automatically prints the name of the function associated with the warning.) [The Lisp machine calls this COMPILER:WARN, approximately.] *BREAK-ON-WARNINGS* Special Variable If non-NIL, WARN prints its message and then goes to the debugger. Continuing causes WARN to return NIL. If *BREAK-ON-WARNINGS* is NIL (the default), then WARN just prints its message and returns NIL. This flag is intended for use when debugging programs that issue warnings and would not be turned on in normal "production." BREAK &optional format-string &rest args Print the message and go directly into the debugger, without allowing any possibility of interception by programmed error-handling facilities. There aren't any error-handling facilities in Common Lisp, but there might be in particular implementations, and there will be in Common Lisp in the future. When continued, BREAK returns NIL. It is permissible to call BREAK with no arguments; it will supply some default message. BREAK is presumed to be used as a way of inserting temporary debugging "breakpoints" in a program, not as a way of signalling errors. Thus continuing from a BREAK would never do anything special, and it does not take the second FORMAT control-string argument that CERROR takes. This and the lack of any possibility of interception by programmed error-handling are the only program-visible differences between BREAK and CERROR. The interactive debugger may choose to display them differently, for instance a CERROR's message might be prefixed with "Error:" and a BREAK's message prefixed with "Break:". This depends on the user-interface style of the particular implementation. Compatibility note: Maclisp's BREAK takes two optional arguments. The first would be a string if Maclisp had strings. The second is a boolean value specifying whether BREAK should break or return immediately. In Common Lisp one makes a BREAK conditional by putting it inside a conditional form such as WHEN or UNLESS. - Specialized error-signalling special forms (or macros) CHECK-TYPE place typespec [string] Special Form Signal an error if the contents of place is not of the desired type. Continuing from this error will ask for a new value, store it into place, and start over, checking the type of the new value and signalling another error if it is still not of the desired type. Subforms of place may be evaluated multiple times, because of the implicit loop generated. CHECK-TYPE returns NIL. place must be a generalized variable reference acceptable to SETF. typespec must be a type expression; it is not evaluated. string must be an English description of the type, starting with an indefinite article (a or an); it is not evaluated. If string is not supplied, it is computed automatically from typespec. The reason that the optional string is allowed is that some usages of CHECK-TYPE may prefer a more specific or more explanatory description of what is wanted than can be generated automatically by the type system. The error message will mention place, its contents, and the desired type. Some implementations may generate a somewhat differently worded error message if they recognize that place is one of the arguments to the function that called CHECK-TYPE. Example: (SETQ X 'FOO) (CHECK-TYPE X (INTEGER 0 *) "a positive integer") Error: The value of X, FOO, is not a positive integer. [The Lisp machine calls this CHECK-ARG-TYPE.] ASSERT test-form [reference]* [string [arg]*] Special Form Signal an error if the value of test-form is false. Continuing from this error will allow the user to alter the values of some variables and will then start over, evaluating the test-form again. ASSERT returns NIL. test-form is any form. Each reference (there may be any number of them, or none) is a generalized-variable reference acceptable to SETF. These are variables that test-form depends on and that it makes sense to allow the user to change when continuing from the error. Subforms of the references are only evaluated if an error is signalled, and may be re-evaluated if the error is signalled again (after continuing without actually fixing the problem). string is an error message string, not evaluated. args are forms evaluated only if an error is signalled, and re-evaluated if the error is signalled again. FORMAT is applied to string and args to get the error message. If string is omitted, a default error message such as "assertion failed" is used; in this case the args must be omitted, since the string serves to delimit the references from the args. The test-form and references are not directly included in the error message, but might be made available for the user's perusal by the debugger. If the user gives the continue command, he should be presented with the opportunity to alter the values of any or all of the references; the details of this depend on each particular implementation's user-interface style, of course. Examples: (ASSERT (VALVE-CLOSED-P V1)) (ASSERT (VALVE-CLOSED-P V1) "Live steam is escaping!") (ASSERT (VALVE-CLOSED-P V1) (VALVE-MANUAL-CONTROL V1) "Live steam is escaping!") (ASSERT (<= MINBASE BASE MAXBASE) BASE "Base ~D is out of the range ~D-~D" BASE MINBASE MAXBASE) ;Note that the user is invited to change BASE, but not the bounds. (ASSERT (= (ARRAY-DIMENSION A 1) (ARRAY-DIMENSION B 0)) A B "The matrix multiplication ~S x ~S cannot be performed" A B) - Exhaustive case analysis special forms (or macros) These macros are included in the standard language, even though a user could write them himself using the other standard facilities provided, for several reasons. It is likely that many people will want these, and so a standard consistent set should be provided rather than allowing a variety of incompatible dialects to develop. The E- versions are just automatically-generated OTHERWISE clauses, but the C- versions require some thought in order to implement correctly, so they should be provided by the system so users don't have to puzzle them out on their own. Individual implementations may be able to do a better job on these, using their own idiosyncratic facilities, than can be done using the standard facilities. Related to this is the fact that if the implementation provides these, they will fit better into its particular user-interface style. There is also the argument from morality: most people are too lazy to put in an otherwise clause to check for cases they didn't anticipate, even if they would agree that they will probably get shafted later. By making this very easy to do (one character) we make it more likely that people will take the trouble to do it. ETYPECASE value [clauses]* Special Form The syntax is the same as TYPECASE, except that no OTHERWISE clause is permitted. If no clause is satisfied, ETYPECASE signals an error with a message constructed from the clauses. It is not permissible to continue from this error. To supply your own error message, use TYPECASE with an OTHERWISE clause containing a call to ERROR. The name of this function stands for either "exhaustive type case" or "error-checking type case". Example: (SETQ X 1/3) (ETYPECASE X (INTEGER (- X)) (SYMBOL (INVERSE X))) Error: The value of X, 1/3, was neither an integer nor a symbol. CTYPECASE reference [clauses]* Special Form The syntax is the same as TYPECASE, except that no OTHERWISE clause is permitted. The reference must be a generalized variable reference acceptable to SETF. If no clause is satisfied, CTYPECASE signals an error with a message constructed from the clauses. Continuing from this error accepts a new value from the user, stores it into reference, and starts over, making the type tests again. Subforms of reference may be evaluated multiple times. The name of this function stands for "continuable exhaustive type case". ECASE value [clauses]* Special Form The syntax is the same as CASE, except that no OTHERWISE clause is permitted. If no clause is satisfied, ECASE signals an error with a message constructed from the clauses. It is not permissible to continue from this error. To supply your own error message, use CASE with an OTHERWISE clause containing a call to ERROR. The name of this function stands for either "exhaustive case" or "error-checking case". Example: (SETQ X 1/3) (ECASE X (ALPHA (FOO)) (OMEGA (BAR))) Error: The value of X, 1/3, is neither ALPHA nor OMEGA. CCASE reference [clauses]* Special Form The syntax is the same as CASE, except that no OTHERWISE clause is permitted. The reference must be a generalized variable reference acceptable to SETF. If no clause is satisfied, CCASE signals an error with a message constructed from the clauses. Continuing from this error accepts a new value from the user, stores it into reference, and starts over, making the clause tests again. Subforms of reference may be evaluated multiple times. The name of this function stands for "continuable exhaustive case". - Issues decided, perhaps by fiat (last chance to change our minds!) Error messages and continue messages are complete sentences starting with a capital letter, thus you can't embed them in the middle of other sentences. No attempt is made to exploit the lack of conjugation of verbs in English; this wouldn't work for other natural languages anyway. Thus we don't say that a continue message could be printed by itself, prefixed by "The continue command will", or prefixed by "Type continue to" in different implementations. *BREAK-ON-WARNINGS* is in. The syntax of ASSERT depends on the use of the error message string as a delimiter, rather than putting an extra level of parentheses around the references. We say (ECASE X (FOO ...) (BAR ...)) rather than (CASE X (FOO ...) (BAR ...) (OTHERWISE-ERROR)) or (CASE X (FOO ...) (BAR ...) OTHERWISE-ERROR). Subforms of the references in CHECK-TYPE, ASSERT, CTYPECASE, and CCASE may be evaluated multiple times, depending on what the implementation wants to do. - Possible future extensions We anticipate that some or all of the following will be added to Common Lisp in the future, when they are better understood. Some of these things exist already as Symbolics extensions to Common Lisp and will be proposed for standardization when things settle down a bit. Should the manual include this list, which may provide some rationale for what is in this first version of the language? A "condition system" providing names for unusual conditions (including but not limited to errors) and a taxonomic system for classifying those names. Extension of FERROR, CERROR, and WARN so that if the first argument is a symbol, it is taken to be a condition name. In this case the format-string is the second argument. If no condition name is specified, a default condition name with no interesting properties is assumed. Ways to define the behavior and relationships of conditions. Ways to use conditions as inter-module interfaces. Ways to use conditions to customize the behavior of the interactive debugger. Ways to establish condition handlers, so that programs may respond to conditions, by throwing out, by continuing, or by correcting the reason for the condition and retrying. Formalization of the notion of "aborting a program," and provision of mechanisms to define where control will be thrown to when the program is aborted, as well as a function to abort the program. This is more complex than it seems because of interaction with the interactive debugger and the condition system. A way to trap errors without going into the debugger, within a certain dynamic scope. Portable debugger details, e.g. TRACE and BREAKON commands. Facilities making it possible for a user to write a portable debugger. Portable floating-point exception, overflow, and rounding control. A way to define the default description string for a user-defined type, used when CHECK-TYPE is called with only two subforms. This can of course be a function of the type's parameters when the type is not simply a symbol.  Received: from MIT-MC by SU-AI with TCP/SMTP; 16 May 83 16:01:25 PDT Received: from SCRC-EUPHRATES by SCRC-TENEX with CHAOS; Mon 16-May-83 19:04:59-EDT Date: Monday, 16 May 1983, 19:02-EDT From: David A. Moon Subject: error.proposal.9 To: Common-Lisp@su-ai This is probably the final version of my error proposal. Unless there are objections it is ready to convert to Scribe, render style-compatible with the Common Lisp manual, and adopt. Our experience is that error-handling of adequate flexibility and power requires a complex error classification system, using flavors or the equivalent. Not everyone in the Common Lisp community agrees, and our implementation of these ideas has only been in existence for half a year, not long enough to consider it mature and stable. Chapter 23 of the laser manual is mainly a rehash of the old Lisp machine condition system (documented in The Lisp Machine Manual), which was generally found to be totally inadequate. Therefore we propose that for the present the language only standardize on ways to SIGNAL errors, not on ways to HANDLE errors. Of course a complete language requires error-handling mechanisms, but many useful portable programs do not require them. Thus it makes sense to defer standardizing this until we understand it better. The rest of this proposal replaces everything in chapter 23 of the laser manual. Most uppercase text should be read as boldface. Italics are not indicated. - Handling errors When an error is signalled, either by calling one of the functions documented in this section or directly by the Lisp system, it is handled in an implementation-dependent way. The general assumption is that each implementation will provide an interactive debugger that prints the error message, along with suitable contextual information such as what function detected the error. The user may interact with the debugger to examine or modify the state of the program in various ways, including abandoning the current computation ("aborting to top level") and continuing from the error. What "continuing" means depends on how the error is signalled; details are given with each error signalling function. [The Lisp machine calls continuing "proceeding".] - General error signalling functions ERROR format-string &rest args Signal an error, with the message constructed by applying FORMAT to the arguments. It is impossible to continue from this kind of error; thus the ERROR function will never return (barring someone meddling with it by redefining it or by using an interactive debugger to force it to return). The error message should not contain a carriage return at the beginning or end, and should not contain any sort of herald indicating that it is an error. The system will take care of these according to whatever its preferred style may be. Conventionally error messages are complete English sentences, ending with a period. Carriage returns in the middle of long messages are okay. There should be no indentation after a carriage return in the middle of an error message. If the debugger in a particular implementation displays error messages indented (e.g. indented by 7 spaces because they are prefixed by "Error: ") then it should take care of inserting the appropriate amount of indentation into a multi-line error message. Similarly, a debugger that prefixes error messages with semicolons should take care of inserting a semicolon at the beginning of each line in a multi-line error message. Note that even within a single implementation, there may be more than one program that presents error messages to the user, and they may use different styles, so the caller of ERROR cannot provide for this in advance. The debugger printout in the following example is not to be taken as a Common Lisp requirement; it is only an example of what an implementation might print when ERROR is called. Example: (ERROR "The command ~S is unrecognized." CMD) Error: The command EMERGECNY-SHUTDOWN is unrecognized. Error signalled by function OPERATOR-COMMAND-EXECUTE. > [The Lisp machine calls this FERROR]. CERROR error-format-string continue-format-string &rest args Signal an error, with the message constructed by applying FORMAT to error-format-string and args. Continuing from such an error will cause CERROR to return NIL. Use CERROR rather than ERROR to signal errors for which you have written recovery code. The name stands for "continuable error," which is too verbose to use for such a common function. A message describing what will happen if the error is continued will be constructed by applying FORMAT to continue-format-string and args. This message should be a one-line description of the effect of continuing. It should be phrased as an English sentence in the imperative mood, just like the documentation of a command or a function. Typically it would be used by an interactive debugger as the documentation of its "continue" command. The message supplied with CERROR should not include any statement of how the "continue" command is given, since this may be different for each debugger. The debugger will insert this into the message, according to its own particular user-interface style. Examples: (UNLESS (= (LIST-LENGTH FORM) 3) (CERROR "Wrong number of arguments in ~S" (IF (< (LIST-LENGTH FORM) 3) "Assume 0 for missing args." "Ignore extra args.") FORM) (SETQ FORM (APPEND FORM '(0 0)))) Error: Wrong number of arguments in (F X) Error signalled by function EXAMPLE. If continued: Assume 0 for missing args. > (DO () ((KNOWN-WORDP X) X) (CERROR "~S is unknown, probably misspelled." "Replace ~S and try again." X) (FORMAT T "~&New word: ") (SETQ X (READ))) ;This example could be done more briefly with ASSERT or ;even with CHECK-TYPE (and SATISFIES). In complex cases where the error message uses some of the args and the continue message uses others of the args, it may be necessary to use the ~G or ~* operators of FORMAT to skip over unwanted arguments in one of the FORMAT control strings. [The Lisp machine calls this FSIGNAL, except that it returns :NO-ACTION rather than NIL and it fails to distinguish between the error message and the continue message.] WARN format-string &rest args Print an error message constructed by applying FORMAT to the arguments, but don't go into the debugger. WARN returns NIL. This would just be FORMAT with output directed to *ERROR-OUTPUT*, except that WARN takes care of advancing to a fresh line before and after the error message and may do other implementation-dependent things (for example, in Symbolics Common Lisp, WARN messages printed during a compilation are associated with the function being compiled and saved for later perusal by editor tools. Furthermore WARN automatically prints the name of the function associated with the warning.) [The Lisp machine calls this COMPILER:WARN, approximately.] *BREAK-ON-WARNINGS* Special Variable If non-NIL, WARN prints its message and then goes to the debugger. Continuing causes WARN to return NIL. If *BREAK-ON-WARNINGS* is NIL (the default), then WARN just prints its message and returns NIL. This flag is intended for use when debugging programs that issue warnings and would not be turned on in normal "production." BREAK &optional format-string &rest args Print the message and go directly into the debugger, without allowing any possibility of interception by programmed error-handling facilities. There aren't any error-handling facilities in Common Lisp, but there might be in particular implementations, and there will be in Common Lisp in the future. When continued, BREAK returns NIL. It is permissible to call BREAK with no arguments; it will supply some default message. BREAK is presumed to be used as a way of inserting temporary debugging "breakpoints" in a program, not as a way of signalling errors. Thus continuing from a BREAK would never do anything special, and it does not take the second FORMAT control-string argument that CERROR takes. This and the lack of any possibility of interception by programmed error-handling are the only program-visible differences between BREAK and CERROR. The interactive debugger may choose to display them differently, for instance a CERROR's message might be prefixed with "Error:" and a BREAK's message prefixed with "Break:". This depends on the user-interface style of the particular implementation. Compatibility note: Maclisp's BREAK takes two optional arguments. The first would be a string if Maclisp had strings. The second is a boolean value specifying whether BREAK should break or return immediately. In Common Lisp one makes a BREAK conditional by putting it inside a conditional form such as WHEN or UNLESS. - Specialized error-signalling special forms (or macros) CHECK-TYPE place typespec [string] Special Form Signal an error if the contents of place is not of the desired type. Continuing from this error will ask for a new value, store it into place, and start over, checking the type of the new value and signalling another error if it is still not of the desired type. Subforms of place may be evaluated multiple times, because of the implicit loop generated. CHECK-TYPE returns NIL. place must be a generalized variable reference acceptable to SETF. typespec must be a type expression; it is not evaluated. string must be an English description of the type, starting with an indefinite article (a or an); it is not evaluated. If string is not supplied, it is computed automatically from typespec. The reason that the optional string is allowed is that some usages of CHECK-TYPE may prefer a more specific or more explanatory description of what is wanted than can be generated automatically by the type system. The error message will mention place, its contents, and the desired type. Some implementations may generate a somewhat differently worded error message if they recognize that place is one of the arguments to the function that called CHECK-TYPE. Example: (SETQ X 'FOO) (CHECK-TYPE X (INTEGER 0 *) "a positive integer") Error: The value of X, FOO, is not a positive integer. [The Lisp machine calls this CHECK-ARG-TYPE.] ASSERT test-form [reference]* [string [arg]*] Special Form Signal an error if the value of test-form is false. Continuing from this error will allow the user to alter the values of some variables and will then start over, evaluating the test-form again. ASSERT returns NIL. test-form is any form. Each reference (there may be any number of them, or none) is a generalized-variable reference acceptable to SETF. These are variables that test-form depends on and that it makes sense to allow the user to change when continuing from the error. Subforms of the references are only evaluated if an error is signalled, and may be re-evaluated if the error is signalled again (after continuing without actually fixing the problem). string is an error message string, not evaluated. args are forms evaluated only if an error is signalled, and re-evaluated if the error is signalled again. FORMAT is applied to string and args to get the error message. If string is omitted, a default error message such as "assertion failed" is used; in this case the args must be omitted, since the string serves to delimit the references from the args. The test-form and references are not directly included in the error message, but might be made available for the user's perusal by the debugger. If the user gives the continue command, he should be presented with the opportunity to alter the values of any or all of the references; the details of this depend on each particular implementation's user-interface style, of course. Examples: (ASSERT (VALVE-CLOSED-P V1)) (ASSERT (VALVE-CLOSED-P V1) "Live steam is escaping!") (ASSERT (VALVE-CLOSED-P V1) (VALVE-MANUAL-CONTROL V1) "Live steam is escaping!") (ASSERT (<= MINBASE BASE MAXBASE) BASE "Base ~D is out of the range ~D-~D" BASE MINBASE MAXBASE) ;Note that the user is invited to change BASE, but not the bounds. (ASSERT (= (ARRAY-DIMENSION A 1) (ARRAY-DIMENSION B 0)) A B "The matrix multiplication ~S x ~S cannot be performed" A B) - Exhaustive case analysis special forms (or macros) These macros are included in the standard language, even though a user could write them himself using the other standard facilities provided, for several reasons. It is likely that many people will want these, and so a standard consistent set should be provided rather than allowing a variety of incompatible dialects to develop. The E- versions are just automatically-generated OTHERWISE clauses, but the C- versions require some thought in order to implement correctly, so they should be provided by the system so users don't have to puzzle them out on their own. Individual implementations may be able to do a better job on these, using their own idiosyncratic facilities, than can be done using the standard facilities. Related to this is the fact that if the implementation provides these, they will fit better into its particular user-interface style. There is also the argument from morality: most people are too lazy to put in an otherwise clause to check for cases they didn't anticipate, even if they would agree that they will probably get shafted later. By making this very easy to do (one character) we make it more likely that people will take the trouble to do it. ETYPECASE value [clauses]* Special Form The syntax is the same as TYPECASE, except that no OTHERWISE clause is permitted. If no clause is satisfied, ETYPECASE signals an error with a message constructed from the clauses. It is not permissible to continue from this error. To supply your own error message, use TYPECASE with an OTHERWISE clause containing a call to ERROR. The name of this function stands for either "exhaustive type case" or "error-checking type case". Example: (SETQ X 1/3) (ETYPECASE X (INTEGER (- X)) (SYMBOL (INVERSE X))) Error: The value of X, 1/3, was neither an integer nor a symbol. CTYPECASE reference [clauses]* Special Form The syntax is the same as TYPECASE, except that no OTHERWISE clause is permitted. The reference must be a generalized variable reference acceptable to SETF. If no clause is satisfied, CTYPECASE signals an error with a message constructed from the clauses. Continuing from this error accepts a new value from the user, stores it into reference, and starts over, making the type tests again. Subforms of reference may be evaluated multiple times. The name of this function stands for "continuable exhaustive type case". ECASE value [clauses]* Special Form The syntax is the same as CASE, except that no OTHERWISE clause is permitted. If no clause is satisfied, ECASE signals an error with a message constructed from the clauses. It is not permissible to continue from this error. To supply your own error message, use CASE with an OTHERWISE clause containing a call to ERROR. The name of this function stands for either "exhaustive case" or "error-checking case". Example: (SETQ X 1/3) (ECASE X (ALPHA (FOO)) (OMEGA (BAR))) Error: The value of X, 1/3, is neither ALPHA nor OMEGA. CCASE reference [clauses]* Special Form The syntax is the same as CASE, except that no OTHERWISE clause is permitted. The reference must be a generalized variable reference acceptable to SETF. If no clause is satisfied, CCASE signals an error with a message constructed from the clauses. Continuing from this error accepts a new value from the user, stores it into reference, and starts over, making the clause tests again. Subforms of reference may be evaluated multiple times. The name of this function stands for "continuable exhaustive case". - Issues decided, perhaps by fiat (last chance to change our minds!) Error messages and continue messages are complete sentences starting with a capital letter, thus you can't embed them in the middle of other sentences. No attempt is made to exploit the lack of conjugation of verbs in English; this wouldn't work for other natural languages anyway. Thus we don't say that a continue message could be printed by itself, prefixed by "The continue command will", or prefixed by "Type continue to" in different implementations. *BREAK-ON-WARNINGS* is in. The syntax of ASSERT depends on the use of the error message string as a delimiter, rather than putting an extra level of parentheses around the references. We say (ECASE X (FOO ...) (BAR ...)) rather than (CASE X (FOO ...) (BAR ...) (OTHERWISE-ERROR)) or (CASE X (FOO ...) (BAR ...) OTHERWISE-ERROR). Subforms of the references in CHECK-TYPE, ASSERT, CTYPECASE, and CCASE may be evaluated multiple times, depending on what the implementation wants to do. - Possible future extensions We anticipate that some or all of the following will be added to Common Lisp in the future, when they are better understood. Some of these things exist already as Symbolics extensions to Common Lisp and will be proposed for standardization when things settle down a bit. Should the manual include this list, which may provide some rationale for what is in this first version of the language? A "condition system" providing names for unusual conditions (including but not limited to errors) and a taxonomic system for classifying those names. Extension of FERROR, CERROR, and WARN so that if the first argument is a symbol, it is taken to be a condition name. In this case the format-string is the second argument. If no condition name is specified, a default condition name with no interesting properties is assumed. Ways to define the behavior and relationships of conditions. Ways to use conditions as inter-module interfaces. Ways to use conditions to customize the behavior of the interactive debugger. Ways to establish condition handlers, so that programs may respond to conditions, by throwing out, by continuing, or by correcting the reason for the condition and retrying. Formalization of the notion of "aborting a program," and provision of mechanisms to define where control will be thrown to when the program is aborted, as well as a function to abort the program. This is more complex than it seems because of interaction with the interactive debugger and the condition system. A way to trap errors without going into the debugger, within a certain dynamic scope. Portable debugger details, e.g. TRACE and BREAKON commands. Facilities making it possible for a user to write a portable debugger. Portable floating-point exception, overflow, and rounding control. A way to define the default description string for a user-defined type, used when CHECK-TYPE is called with only two subforms. This can of course be a function of the type's parameters when the type is not simply a symbol.  Received: from MIT-MC by SU-AI with TCP/SMTP; 16 May 83 01:01:15 PDT Date: 16 May 1983 04:04 EDT From: Kent M. Pitman To: Moon @ SCRC-TENEX cc: Common-Lisp @ SU-AI, JWalker @ SCRC-TENEX I did in fact propose that :keywords be used in lambda lists years ago and at that time there was a good argument against it: such variables could be bound and so it wasn't fair to special case them. Only with the decision last August of making :keywords self-evaluate has this been a possible course of action. George Carrette mentioned to me today that the NIL interpreter has to go checking the pnames of symbols to see if they start with "&" every time you bind a symbol. That seems pretty gross. Checking the package cell for EQness would be slightly faster or at least more elegant. In fact, the argument about :OTHERWISE-ERROR and COND was slightly false in that it's not inconsistent to allow keywords in COND now that you can know what they will EVAL to. eg, (COND (FOO X) (:OTHERWISE Y)) (COND (FOO X) (:OTHERWISE-ERROR Y)) would be specially recognized syntaxes. You'd case on the surface syntax, not on what FOO evaluates to, so if FOO evaluted to :OTHERWISE or :OTHERWISE-ERROR, that'd just mean true. It's a plausible model of the world. It would be more regular and would unify a lot of concepts that Common Lisp has been drifting toward. But I've had my say and didn't really want to slow down anyone's work. As I said, I was just thinking aloud. I'll consider the issue closed for now unless someone else re-opens it...  Received: from MIT-MC by SU-AI with TCP/SMTP; 16 May 83 00:39:53 PDT Date: Monday, 16 May 1983 02:47-EDT From: MOON at SCRC-TENEX To: Kent M. Pitman Cc: Common-Lisp at SU-AI, JWalker at SCRC-TENEX Subject: Keywords, :Keywords, &More-Keywords In-reply-to: The message of 15 May 1983 14:40 EDT from Kent M. Pitman OTHERWISE is not a keyword for the same reason that DECLARE and LAMBDA are not keywords. While it isn't a function, and it isn't a variable, it's still a syntactic word of the Lisp language. Or maybe it just would look ugly with a colon. Whether it should be &OPTIONAL or :OPTIONAL is an entirely unrelated issue. Certainly the reasons why these are different are historical. I can't defend the choice of &OPTIONAL except to say that if you were serious about changing this you should have brought it up two years ago, not a week or two before the final first version of the manual comes out. Guy certainly chose the quotation at the front of the manual rather well.  Received: from CMU-CS-C by SU-AI with TCP/SMTP; 15 May 83 22:05:26 PDT Received: ID ; 16 May 83 01:05:28 EDT Date: Mon, 16 May 1983 01:05 EDT From: Scott E. Fahlman To: common-lisp@su-ai Subject: issues The next edition of the manual is nearing final-assembly-and-test. It is my belief that we have more or less converged on Moon's proposals for DEFSETF and for the ERROR chapter. At least, several key people have endorsed this proposal and nobody has raised major objections. Unless someone objects soon and violently (and I hope nobody does), Guy plans to put these into the next edition of the manual, for final review by the critics' gallery. With a little luck, maybe we will have converged sufficiently on the new package proposal to get that in too, though we're not there as of right now, but maybe my latest proposal will make it. -- Scott  Received: from CMU-CS-C by SU-AI with TCP/SMTP; 15 May 83 21:48:38 PDT Received: ID ; 16 May 83 00:48:32 EDT Date: Mon, 16 May 1983 00:48 EDT From: Scott E. Fahlman To: common-lisp@su-ai Subject: packages, yet again One more try. I guess if people always knew what they were getting into, nobody would ever try to do anything interesting. Major new features of this proposal: 1. The old "package augmenting" DECLARE-PACKAGE is gone. Flexible, but deemed to be too dangerous, since it allows users of a package to mess with that package's external symbols. 2. INCLUDE-PACKAGE is gone. If you want to do what INCLUDE-PACKAGE used to do, you call USE-PACAKGE on the other package, then explictly EXPORT all of the symbols that you want to pass along. Deemed to make your intentions clearer and to be less confusing overall. 3. If IMPORT or USE-PACAKGE creates a name conflict between one or more of the imported symbols and some distinct symbol available in the current package, a correctable error is signalled. Each package keeps a list of symbols that have been explicitly declared to be present and to shadow other symbols (SHADOW and SHADOWING-INPUT add symbols to this list) and these symbols always win any conflicts without signalling an error. 4. A package must be created and all its externals must be set up before any other package uses it. If a package is being used by any other package, an error is signalled if you try to change its set of external symbols in any way. 5. The PROVIDE and REQUIRE functions from the Laser edition are suggested as a way for the user to load complicated things without having to worry about complicated order-of-load restrictions. A few changes are proposed in these functions. They probably want to be moved into this section, since there is a strong interaction with packages. (Do we really want to keep these separate, or would a merger of modules and packages make sense?) The new version is relatively simple, fairly safe, and probably more restrictive than it needs to be. At this point it seems better to go for something simple and restricitve that might conceivably survive the onslaught of the critics. We can always pass the user more rope later. Here we go again: *************************************************************************** @chapter [Packages] @section [Overview] One problem with earlier Lisp systems is the use of a single name space for all symbols. In large Lisp systems, with modules written by many different programmers, accidental name collisions become a serious problem. Common Lisp addresses this problem through the @i[package system], derived from an earlier package system developed for Lisp Machine Lisp. In addition to preventing name-space conflicts, the package system makes the modular structure of large Lisp systems more explicit. A @i[package] is a data structure that establishes a mapping from print names (strings) to symbols. The package thus replaces the ``oblist'' or ``obarray'' machinery of earlier Lisp systems. At any given time one package is current, and this package is used by the reader in translating strings into symbols. The current package is always the value of the global variable @b[*package*]. It is possible to refer to symbols in packages other than the current one through the use of @i[package qualifiers] in the symbol reference. For example @b[foo:bar] refers to the symbol whose name is @b[bar] in the package whose name is @b[foo]. The string-to-symbol mappings available in a given package are divided into two classes, @i[external] and @i[internal]. We refer to the symbols accessible via these mappings as being @i[external] and @i[internal] symbols of the package in question, though really it is the mappings that are different and not the symbols themselves. Within a given package, a name refers to one symbol or to none; if it does refer to a symbol, that symbol is either external or internal in that package, but not both. External symbols are part of the package's public interface to other packages. These are supposed to be chosen with some care and are advertised to users of the package. Internal symbols are for internal use only, and these symbols are normally hidden from other packages. Most symbols are created as internal symbols; they become external only if they appear explicitly in some package's @b[export] command. A symbol may appear in many packages. It will always have the same print name wherever it appears, but it may be external in one package and internal in another. A name may refer to different symbols in different packages. Packages may be built up in layers. From the point of view of a package's user, the package is a single collection of mappings from strings into internal and external symbols. However, some of these mappings may be established within the package itself, while other mappings are inherited from other packages via the @b[use-package] construct. (The mechansims responsible for this inheritance are described below.) In what follows, we will refer to a symbol as being "accessible" in a package if it can be referred to in that package without a qualifier, regardless of whether the mapping occurs within that package or via inheirtance; we will refer to a symbol as being "present" in a package if the mapping is in the package itself and is not inherited from somewhere else. @section [Consistency Rules] Package-related bugs can be very subtle and confusing: things are not what they appear to be. The Common Lisp package system is designed with a number of safety features to prevent most of the common bugs that would otherwise occur in normal use. This may seem over-protective, but experience with earlier package systems has shown that such safety features are needed. In dealing with the package system, it is useful to keep in mind the following consistency rules, which remain in force as long as the value of @b[*package*] is not changed by the user or his code: @begin [itemize] @b[Read-Read consistency:] Reading the same print name always gets you the same (EQ) symbol. @b[Print-Read consistency:] An interned symbol always prints as a sequence of characters which, when read back in, yields the same (EQ) symbol. @b[Print-Print consistency:] If two interned symbols are not EQ, then their printed representations will not be the same sequence of characters. @end [itemize] These consistency rules remain true in spite of any amount of implicit interning caused by typing in Lisp forms, loading files, etc. This has the important implication that results are reproducible regardless of the order of loading files or the exact history of what symbols were typed in when. The rules can only be violated by explicit action: changing the value of @b[*package*], forcing some action by continuing from an error, or calling one of the ``dangerous'' functions: @b[unintern], @b[shadow], or @b[shadowing-import]. In general, an error is signalled if the user takes any action that would replace one symbol available in a package with a different symbol of the same name. Such "implicit shadowing" is usually an error. However, the user can explictily ask for such shadowing to occur with the @b[shadow] and @b[shadowing-import] functions. These functions indicate that certain symbols are to be directly present in a package and that they are to take precedence over any symbol of the same name that would otherwise be inherited by that package. Each package keeps a list of such shadowing symbols, so that subsequent calls to @b[use-package] or @b[import] will not interfere with the user's declared intentions. @section [Package Names] Each package has a name (a string) and perhaps some nicknames. These are assigned when the package is created, though they can be changed later. A package's name should be something long and self-explanatory like "editor"; there might be a nickname that is shorter and easier to type, like "ed". Package name strings retain whatever mixture of upper and lower case characters the user supplies, but the translation from names to packages is case-insensitive, so users can generally refer to packages without worrying about case. There is a single name space for packages. The function @b[find-package] translates a package-name or nickname into the associated package. The function @b[package-name] returns the name of a package. The function @b[package-nicknames] returns a list of all nicknames for a package. The function @b[rename-package] removes a package's current name and nicknames and replaces them with new ones specified by the user. Package renaming is occasionally useful when, for development purposes, it is desirable to load two versions of a package into the same Lisp. We can load one, rename it, and then load the other, without getting a lot of name conflicts. @section [Translating Strings to Symbols] The value of the special variable @b[*package*] must always be a package object (not a name). This is referred to as the @i[current package]. When the Lisp reader has, by parsing, obtained a string of characters thought to name a symbol, that name is looked up in the current package. This lookup may involve looking in other packages whose external symbols are inherited by the current package (see below). If the name is found, the corresponding symbol is returned. If the name is not found, a new symbol is created for it and is placed in the current package as an internal symbol; if the name is seen again while this same package is current, the same symbol will then be returned. When a new symbol is created, a pointer to the package in which it is initially placed is stored in the @i[package cell] of that symbol; the package is said to be the symbol's @i[home package], and is said to @i[own] the symbol. (Some symbols are not owned by any package; they are said to be @i[uninterned], and their package cell contains NIL.) Often it is desirable to refer to an external symbol in some package other than the current one. This is done through the use of a @i[qualified name], consisting of a package name, then a colon, then the print name of the symbol. This causes the symbol's name to be looked up in the specified package, rather than in the current one. For example, ``@b[editor:buffer]'' refers to the external symbol named ``@b[buffer]'' available in the package named ``@b[editor]'', regardless of whether there is a symbol named ``@b[buffer]'' in the current package. If there is no package named ``@b[editor]'', or if no symbol named ``@b[buffer]'' is available in ``@b[editor]'' or if ``@b[buffer]'' is an internal symbol in ``@b[editor]'', a correctable error is signalled to ask the user what he really wants to do. On rare occasions, a user may need to refer to an @b[internal] symbol of some package other than the current one. It is illegal to do this with the colon qualifier, since accessing an internal symbol of some other package is usually a mistake. However, this operation is legal if you use ``@b[#:]'' as the separator in place of the usual colon. If ``@b[editor#:buffer]'' is seen, the effect is exactly the same as reading ``@b[buffer]'' with @b[*package*] temporarily rebound to the package whose name is ``@b[editor]''. This special-purpose qualifier should be used with caution. The package named @b[keyword] contains all keyword symbols used by the Lisp system itself and by user-written code. Such symbols must be easily accessible from any package, and name conflicts are not an issue since these symbols are used only to label arguments and never to carry package-specific values or properties. Because keyword symbols are used so frequently, Common Lisp provides a special reader syntax for them: any symbol preceded by a colon but no package name (for example ``@b[:foo]'') is added to (or looked up in) the @b[keyword] package as an @i[external] symbol. The @b[keyword] package is also treated specially in that whenever a symbol is added to the @b[keyword] package, the symbol is automatically declared to be a constant and is made to have itself as its value. This is why every keyword evaluates to itself. As a matter of style, keywords should always be accessed using the leading-colon convention; you never want to import or inherit keywords into any other package. Each symbol contains a package slot that is used to record the home package of the symbol, or NIL if the symbol is uninterned. When an interned symbol is printed, if it is an external symbol in the keyword package then it is printed with a preceding colon; otherwise, if it is available (directly or by inheritance) in the current package, it is printed without any qualification; otherwise, it is printed with the name of the home package as the qualifier, using ``@b[:]'' as the separator if the symbol is external and ``@b[#:]'' if not. A symbol that is uninterned (has no home package) is printed preceded by ``@b[#:]''. It is possible, by the clever use of import and unintern, to create a symbol that has no recorded home package, but that in fact is interned in some package. The system does not check for this. In summary, the following four cases are defined: @Begin[describe] @b[foo:bar] @\Look up "bar" among the external symbols available in the package named "foo". @b[foo#:bar] @\Look up "bar" among the external and internal symbols available in the package named "foo". @b[:bar] @\The symbol named "bar" is a self-evaluating keyword. @b[#:bar] @\The symbol named "bar" is uninterned. @End[describe] All other uses of colons within names of symbols are not defined by Common Lisp, but are reserved for implementation-dependent use; this includes names that end in a colon, contain two or more colons, or consist of just a colon. @section [Exporting and Importing Symbols] Symbols from one package may be made available in another package in two ways. First, any individual symbol may be added to a package by use of the @b[import] function. The form @b[(import 'editor:buffer)] takes the external symbol named @b[buffer] in the @b[editor] package (this symbol was located when the form was read by the Lisp reader) and adds it to the current package as an internal symbol. The imported symbol is not automatically exported from the current package, but if it is already present and external, that is not changed. After the call to @i[import] it is possible to refer to @b[buffer] in the importing package without any qualifier. The status of @b[buffer] in the package named @b[editor] is unchanged, and @b[editor] remains the home package for this symbol. Once imported, a symbol is @i[present] in the importing package and can only be removed by calling @b[unintern]. If the symbol is already present in the importing package, @b[import] has no effect. If a distinct symbol with the name @b[buffer] is available in the importing package (directly or by inheritance) a correctable error is signalled. The user will then be offered a choice of whether or not the importation of this symbol is to proceed. If the user really wants to do a shadowing import without getting an error, he should use the @b[shadowing-import] function. This inserts the symbol into the specified package as an internal symbol, regardless of whether another symbol of the same name will be shadowed by this action. The symbol is also added to the package's shadowing-symbols list. @b[Shadowing-import] may create confusing situations which violate the consistency rules, so it should be used with caution. The second mechanism is provided by the @b[use-package] function. This causes a package to inherit all of the external symbols of some other package. These symbols become available as @i[internal] symbols of the using package. That is, they can be referred to without a qualifier while this package is current, but they are not passed along to any other package that uses this package. Often a user, working by default in the @b[user] package, will load a number of packages into his Lisp to provide an augmented working environment; then he will call @b[use-package] on each of these packages so that he can easily access their external symbols. When @b[use-package] is called, it checks the external symbols of the package being used against the set of symbols already available in the using package. If there is a conflict -- that is, if one of the imported symbols has the same name as a distinct symbol already present in the package -- then a correctable error is signalled asking the user whether the imported or the existing symbol is to be retained in the package. This error does not occur if the existing symbol is on the package's shadowing-symbols list; in that case, the existing symbol wins and continues to be available in that package, shadowing the symbol from the package being used. Thus, if the user wants to use a package, minus a few of its external symbols (for which he has local versions), he can avoid any error by explictly shadowing those symbols before calling @b[use-package]. @b[Unuse-package] undoes the effects of a previous @b[use-package]. The external symbols of the used package are no longer inherited. However, any symbols that have been imported into the using package continue to be present in that package. There is no way to inherit the @i[internal] symbols of another package; to refer to an internal symbol, you must either make that symbol's home package current, use a qualifier, or import that symbol into the current package. When @b[intern] or some other function wants to look up a symbol in a given package, it first looks for the symbol among the external and internal symbols of the package itself; then it looks through the external symbols of the used packages in some unspecified order. The order does not matter; according to the rules above if conflicting symbols appear in two or more packages inherited by package X, a symbol of this name must also appear in X itself as a shadowing symbol. Of course, implementations are free to choose other, more efficient ways of implementing this search, as long as the user-visible behavior is equivalent to what is described here. The @b[export] function takes a symbol that is available in some package and makes it an external symbol of that package. If the symbol is already available as an external symbol in the package, @b[export] has no effect. If the symbol is directly present in the package as an internal symbol, it is simply changed to external status. If it is available as an internal symbol via @b[use-package], the symbol is first imported into the package, then exported. (It is now present in this package whether or not it continues to use the symbol's original package.) If the symbol is not available at all in the specified package, a correctable error is signalled which, upon continuing, asks the user whether the symbol should be imported. The @b[unexport] function is provided mainly as a way to undo erroneous calls to @b[export]. It works only on symbols that are directly present in the current package, switching them back to internal status. If @b[unexport] is given a symbol that is already available as an internal symbol in the current package, it does nothing; if it is given a symbol that is not available in the packge at all, it signals an error. @Section[Built-in Packages] The following packages are built into the Common Lisp system: @begin[description] @b[lisp]@\The package named @b[lisp] contains the primitives of the Common Lisp system. Its external symbols include all of the user-visible functions and global variables that are present in the Common Lisp system, such as @b[car], @b[cdr], @b[*package*], etc. Almost all other packages will want to use @b[lisp] so that these symbols are available without qualification. @b[user]@\The @b[user] package is, by default, the current package at the time a Common Lisp system starts up. This package uses the @b[lisp] package. @b[keyword]@\This package contains all of the keywords used by built-in or user-defined Lisp functions. Symbols that start with a colon are treated as external symbols in the this package. All symbols in this package are treated as constants that evaluate to themselves, so the user can type @b[:foo] instead of @b[':foo]. @b[system]@\This package name is reserved to the implementation. Normally this is used to contain names of implementation-dependent system-interface functions. This package uses @b[lisp] and has the nickname @b[sys]. @end[description] @Section[Package System Functions and Variables] Some of the following have been described earlier, but are included here for completeness. Note that the following functions are meant to be used at top-level in files, and should only be used at top-level if you want to guarantee proper compilation across all Common Lisp implemenations: @b[in-package], @b[import], @b[export], @b[unexport], @b[shadow], @b[use-package], and @b[unuse-package]. For the functions described here, all optional arguments named @i[package] default to the current value of @b[*package*]. Where a function takes an argument that is either a symbol or a list of symbols, an argument of NIL is treated as an empty list of symbols. @Defvar[Var {*package*}] The value of this variable must be a package; this package is said to be the current package. The initial value of @b[*package*] is the @b[user] package. The @b[load] function rebinds @b[*package*] to its current value. If some form in the file changes the value of @b[*package*] during loading, the old value will be restored when the loading is completed. @Enddefvar @Defun[Fun {make-package}, Args {@i[package-name] @key @i[nicknames] @i[use]}] Creates and returns a new package with the specified package name. The @b[:nicknames] argument must be a list of strings to be used as alternative names for the function. These names must not conflict with any existing package names; if they do, a correctable error is signalled. The @b[:use] argument is a list of packages or package names whose external symbols are to be inherited by the new package. These package must already exist, and their external symbols must already have been exported. If not supplied, @i[:use] defaults to a list of one package, the @b[lisp] package. @Enddefun @Defun[Fun {in-package}, Args {@i[package-name] @key @i[nicknames] @i[use]}] The @b[in-package] function is intended to be placed at the start of a file containing a subsystem that is to be loaded into some package other than @b[user]. It is similar to @b[make-package], except that after the new package is created, @b[*package*] is set to it. This binding will remain in force until changed by the user (perhaps with another @b[in-package] call), or until the @b[*package*] variable reverts to its old value at the completion of a @b[load] operation. @enddefun @Defun[Fun {find-package}, Args {@i[name]}] The @i[name] must be a string that is the name or nickname for a package. Returns the corresponding package, or NIL if no such package exists. This match ignores case (as in @b[string-equal]). @Enddefun @Defun[Fun {package-name}, Args {@i[package]}] The argument must be a package. This function returns the string that names that package. @Enddefun @Defun[Fun {package-nicknames}, Args {@i[package]}] The argument must be a package. This function returns the list of nickname strings for that package, not including the primary name. @Enddefun @Defun[Fun {package-use-list}, Args {@i[package]}] Returns the list of other packages used by the argument package. @Enddefun @Defun[Fun {package-used-by-list}, Args {@i[package]}] Returns the list of other packages that use the argument package. @Enddefun @Defun[Fun {package-shadowing-symbols}, Args {@i[package]}] Returns the list of symbols in that have been declared as shadowing symbols in this package by @b[shadow] or @b[shadowing-import]. All symbols on this list are present in the specified package. @Enddefun @Defun[Fun {list-all-packages}, Args {}] This function returns a list of all packages that currently exist in the Lisp system. @Enddefun @Defun[Fun {intern}, Args {@i[string] @optional @i[package]}] @i[Package], which defaults to the current package, is searched for a symbol with the name specified by the @b[string] argument. This search will include inherited symbols, as described above. If a symbol with the specified name is found, it is returned. If no such symbol is found, one is created and is installed in the current package as an internal symbol. This symbol is returned. @Incompatibility{Conceptually, @b[intern] translates a string to a symbol. In Maclisp and several other Lisps, @b[intern] can take either a string or a symbol as its argument; in the latter case, the symbol's print name is extracted and used as the string. However, this leads to some confusing issues about what to do if @b[intern] finds a symbol that is not @b[eq] to the argument symbol. To avoid such confusion, we require the argument to be a string.} @Enddefun @Defun[Fun {find-symbol}, Args {@i[string] @optional @i[package]}] This is identical to @b[intern], but it never creates a new symbol. If a symbol with the specified name is found in the current package, directly or by inheritance, the symbol found is returned. The second return value is T, indicating that the symbol was found. The third value is T if the symbol is an external symbol in the specified package, NIL if it is internal. If the symbol is not found in the specified package, @b[find-symbol] returns NIL for all three values. @Enddefun @Defun[Fun {find-external-symbol}, Args {@i[string] @optional @i[package]}] This is identical to @b[find-symbol], but it does not look for internal symbols even in @i[package] itself. This is the type of search done by the reader for symbols qualified with a package name and a colon. The return values are the same as for @b[find-symbol], with the third return value always being T (external symbol) if a symbol is found. @Enddefun @Defun[Fun {unintern}, Args {@i[symbol] @optional @i[package]}] If the specified symbol is present in the specified package, it is removed from this package, and also from the package's shadowing-symbols list if it is present there. Moreover, if @i[package] is the home package for the symbol, the symbol is made to have no home package. @b[Unintern] returns T if it actually removed a symbol, and NIL otherwise. Note that @i[unintern] should be used with caution, since it can produce confusing situations that violate the consistency rules listed above. @Incompatibility{The equivalent of this in @maclisp is @f[remob].} @Enddefun @Defun[Fun {export}, Args {@i[symbols] @optional @i[package]}] The @i[symbols] argument should be a list of symbols, or possibly a single symbol. These symbols become available as external symbols in the current package. (See the previous section for details.) If @i[package] is currently used by another package, any use of @i[export] signals a correctable error. Returns T. By convention, a call to @f[export] listing all exported symbols is placed near the start of a file to advertise which of the symbols mentioned the file are intended to be used by other programs. @Enddefun @Defun[Fun {unexport}, Args {@i[symbols] @optional @i[package]}] The argument should be a list of symbols, or possibly a single symbol. These symbols become internal symbols in @i[package]. (See the previous section for details.) If @i[package] is currently used by another package, any use of @i[unexport] signals a correctable error. Returns T. @Enddefun @Defun[Fun {import}, Args {@i[symbols] @optional @i[package]}] The argument should be a list of symbols, or possibly a single symbol. These symbols become internal symbols in @i[package], and can therefore be referred to without a colon qualifier. @b[Import] signals a correctable error if any of the imported symbols has the same name as some distinct symbol already available in the package. (See the previous section for details.) Returns T. @Enddefun @Defun[Fun {shadowing-import}, Args {@i[symbols] @optional @i[package]}] This is like import, but it does not signal an error even if the importation of a symbol would shadow some symbol already available in the package. In addition to being imported, the symbol is placed on the shadowing-symbols list of @i[package]. (See the previous section for details.) Note that @i[shadowing-import] should be used with caution, since it can produce confusing situations that violate the consistency rules listed above. Returns T. @Enddefun @Defun[Fun {shadow}, Args {@i[symbols] @optional @i[package]}] The argument should be a list of symbols, or possibly a single symbol. The print-name of each symbol is extracted, and the current package is searched for a symbol of that name. If such a symbol is present in this package (directly, not by inheritance) then nothing is done. Otherwise, a new symbol is created with this print name, and it is inserted in the current package as an internal symbol. The symbol is also placed on the shadowing-symbols list of @i[package]. Note that @i[shadow] should be used with caution, since it can produce confusing situations that violate the consistency rules listed above. Returns T. @Enddefun @Defun[Fun {use-package}, Args {@i[packages]}] The argument should be a list of packages or package names, or possibly a single package or package name. These packages are added to the the package's use-list if they are not there already. All external symbols in the argument packages become available in the current package as internal symbols. (See the previous section for details.) Returns T. @enddefun @Defun[Fun {unuse-package}, Args {@i[packages]}] The argument should be a list of packages or package names, or possibly a single package or package name. These packages are removed from the current package's use-list. Returns T. @enddefun @Defun[Fun {find-all-symbols}, Args {@i[string-or-symbol]}] Searches every package in the Lisp system for symbols whose print-name is the specified string, and returns a list of such symbols. This search is case-sensitive. If the argument is a symbol, its print-name supplies the string to be searched for. @enddefun @Defmac[Fun {do-symbols}, Args {(@i[var] @Mopt<@i[package]> @Mopt<@i[result-form]>) @Mstar<@i[declaration]> @mstar<@i[tag] @mor @i[statement]>}] @b[Do-symbols] provides straightforward iteration over the symbols of a package. The body is performed once for each symbol available in the @i[package], in no particular order, with the variable @i[var] bound to the symbol. Then @i[resultform] (a single form, @i[not] an implicit @b[progn]) is evaluated, and the result is the value of the @b[do-symbols] form. (When the @i[resultform] is evaluated, the control variable @i[var] is still bound, and has the value @nil.) If @i[resultform] is omitted, the result is NIL. @b[Return] may be used to terminate the iteration prematurely. If execution of the body affects which symbols are contained in the @i[package], other than possibly to remove the symbol currently the value of @i[var] by using @b[unintern], the effects are unpredictable. @Enddefmac @Defmac[Fun {do-external-symbols}, Args {(@i[var] @Mopt<@i[package]> @Mopt<@i[result-form]>) @Mstar<@i[declaration]> @mstar<@i[tag] @mor @i[statement]>}] @b[Do-external-symbols] is just like @b[do-symbols], except that only the external symbols of the specified package are scanned. @Enddefmac @Defmac[Fun {do-all-symbols}, Args {(@i[var] @Mopt<@i[result-form]>) @Mstar<@i[declaration]> @mstar<@i[tag] @mor @i[statement]>}] This is similar to @b[do-symbols], but executes the body once for every symbol contained in every package. (This may not get all symbols whatsoever, depending on the implementation.) It is @i[not] in general the case that each symbol is processed only once, since a symbol may appear in many packages. @Enddefmac @Section[Modules] A @i[module] is a Common Lisp subsystem that is loaded from one or more files. A module is normally loaded as a single unit, regardless of how many files are involved. A module may consist of one package or several packages. The file-loading process is necessarily implementation-dependent, but Common Lisp provides some very simple portable machinery for naming modules, keeping track of which modules have been loaded, and for loading modules as a unit. @Defvar[Var {*modules*}] @Defun[Fun {provide}, Args {@i[module-name]}] @Defun1[Fun {require}, Args {@i[module-name] @optional @i[pathname]}] Each module has a unique name (a string). If the module consists of a single package, it is customary for the package and module names to be the same. The variable @b[*modules*] is a list of names of the modules that have been loaded into the Lisp system so far. The @b[provide] function adds a new module name to this list, thereby indicating that the module in question has been loaded. The @b[require] function tests whether a module is already present (using a case-insensitive comparison) and, if not, loads the appropriate file or set of files. The pathname argument, if present, is a single pathname or a list of pathenames whose files are to be loaded in order, left to right. If the pathname argument is NIL or is not provided, the system will attempt to determine which files to load in some system-dependent manner. This will typically involve some central registry of module names and the associated file-lists. @section [An Example] Most users will want to load and use packages but will never need to build one. Often, a user will load a number of packages into the @b[user] package whenever he uses Common Lisp. Most implementations will provide some sort of "init file" mechanism to make such setup automatic when the Lisp starts up. Init files are not normally compiled. @begin [Lisp] ;;; Lisp init file for I. Newton. ;;; Set up the USER package the way I like it. ;;; Calculus is used a lot. Get easy access to all its symbols. (require "calculus") (use-package "calculus") ;;; Same for Newtonian mechanics. (require "newtonian-mechanics") (use-package "newtonian-mechanics") ;;; I just want a few thing from here, and other things conflict. ;;; Import just what I need into the USER package. (require "relativity") (import '(relativity:speed-of-light relativity:ignore-small-errors)) ;;; These are worth loading, but I will use qualifiers to get at any ;;; symbols I need. (require "phlogiston.lsp") (require "alchemy.lsp") @end [Lisp] When each of two files uses some symbols from the other, we must be careful to put the contents of the file in the file in the proper order. Typically each file contains a single package that is a complete module. The contents of such a file should include the following items, in order: @Begin[enumerate] A @b[provide] call that announces the module name. An @b[in-package] call that establishes the package. A @b[shadow] call that establishes any local symbols that will shadow symbols that would otherwise be inherited from packages that this package will use. An @b[export] call that establishes all of this package's external symbols. Any number of @b[require] calls to load other modules which the contents of this file might want to use or refer to. By placing these @b[requires] calls here, we make it possible for the packages being loaded to refer to external symbols in this package. Any number of @b[use-package] and @b[import] calls, to make it the symbols from other packages available in this package. Finally, the definitions making up the contents of this package/module. @End[enumerate] << Anybody got a good mnemonic for the sequence "PISERIUC"? All I can come up with is "Please insert seven extra rubles in user consultant." Not too good. >> Now, suppose that the @b[phlogiston] and @b[alchemy] packages are single-file, single-package modules as described above. Phlogiston wants to use the alchemy package, and alchemy wants to use several external symbols from phlogiston. The following definitions allow the user to supply @b[require] statement for either of these modules, or for both of them in either order. The file "alchemy.lisp": @begin [lisp] ;;; Alchemy functions, written and maintained by Merlin, Inc. ;;; Both the module and the package are named "alchemy". (provide "alchemy") (in-package "alchemy") ;;; Nothing to shadow. ;;; Here is our external interface. (export '(lead-to-gold gold-to-lead antimony-to-zinc elixir-of-life)) ;;; This file uses some functions from the phlogiston package/module. (require "phlogiston") ;;; We use this a lot, so import it. It's external in phlogiston package. (import '(phlogiston:make-fire-bottle)) ;;; Now for the real contents of this file. (defun lead-to-gold (x) "Takes a quantity of lead, returns gold." (when (> (phlogiston:heat-flow x) 3) ; Using a qualified symbol. (make-fire-bottle x)) ; Using an imported symbol. (gild x)) ;;; And so on ... @end [lisp] The file "phlogiston.lisp": @begin [lisp] ;;; Phlogiston functions, written and maintained by Thermofluidics, Ltd. ;;; Both the module and the package are named "phlogiston". (provide "phlogiston") (in-package "phlogiston") ;;; Nothing to shadow. ;;; Here is our external interface. (export '(heat-flow cold-flow mix-fluids separate-fluids burn make-fire-bottle)) ;;; This file uses some functions from the alchemy package/module. (require "alchemy") ;;; We use alchemy functions a lot, so use the package. (use-package "alchemy") ;;; The real contents of this package/module. (defun heat-flow (amount x y) "Make some amount of heat flow from x to y." (when feeling-weak (quaff (elixir-of-life))) ; No qualifier needed. (push-heat amount x y)) ;;; And so on ... @end [lisp] For very large packages whose contents are spread over several files (the @b[lisp] package is an example), it is recommended that the author create the package and declare all of the shadows and external symbols in a separate file, so that this can be loaded before anything which might use symbols from this package.  Received: from MIT-MC by SU-AI with TCP/SMTP; 15 May 83 12:56:25 PDT Date: Sunday, 15 May 1983, 15:57-EDT From: KMP@MIT-MC Sender: kbe@MIT-OZ Subject: [Griss@UTAH-20: Forwarded] To: "Common-Lisp%SU-AI"@MIT-MC Cc: JWalker@SCRC-TENEX Date: 15 May 1983 1310-MDT From: Martin.Griss Subject: Re: Keywords, :Keywords, &More-Keywords To: KMP@MIT-MC cc: Griss@UTAH-20 In-Reply-To: Your message of 15-May-83 1440-MDT That certainly has a LOT of appeal to me! M -------  Received: from MIT-MC by SU-AI with TCP/SMTP; 15 May 83 11:37:46 PDT Date: 15 May 1983 14:40 EDT From: Kent M. Pitman Subject: Keywords, :Keywords, &More-Keywords To: Moon @ SCRC-TENEX cc: Common-Lisp @ SU-AI, JWalker @ SCRC-TENEX I don't have a strong opinion on the ECASE vs keyworded-CASE. I just wanted to make sure the alternatives were looked on. The tension is basically one of an extended function name space vs a keyworded syntax; Common Lisp has tended to prefer keywords, so I figured it was best to at least review the keyword alternative. The issue you bring up about OTHERWISE is an interesting one. I wonder if OTHERWISE -should- be (or "should have been") a keyword. I have always disliked its being a global symbol just so it could be used in this peculiar case. Also, users may believe it is keyword-like in nature and be very confused by the fact that :OTHERWISE doesn't work! Already there is a big problem with users not understanding why we've chosen &keywords and :keywords. The way keywords are currently defined, making the words in the bvl be :keywords would not be incompatible since it should be an error to bind a symbol in the keyword package anyway! I am not really proposing a change; just thinking aloud. We will some day be subject to a lot of flack from users who claim that the spec is cluttered by more layers of syntax than it need be. Consider that a novice might find (DEFUN FOO (X :OPTIONAL Y Z :REST W) ...) significantly easier to learn. And even (SELECTQ FOO ((A B) 3) ((C) 4) (:OTHERWISE 5)) has a certain appeal... I suppose the :OPTIONAL change might really throw LispM Lisp through a loop during any transition ... On the other hand, I bet changing all OTHERWISEs, &RESTs, &OPTIONALs, etc. would be as simple as a single Tags Query Replace ... And once you release the language, a change like that will never be so easy again ... Oh well. I think if someone else did take this discussion seriously that I probably would too... --kmp  Received: from MIT-MC by SU-AI with TCP/SMTP; 14 May 83 23:08:10 PDT Received: from SCRC-EUPHRATES by SCRC-TENEX with CHAOS; Sun 15-May-83 01:15:32-EDT Date: Sunday, 15 May 1983, 02:06-EDT From: David A. Moon Subject: New Error Proposal: suggestions to be incorporated To: Scott E. Fahlman Cc: Common-Lisp@SU-AI In-reply-to: The message of 15 May 83 01:28-EDT from Scott E. Fahlman I will mail out an updated version of my error proposal tomorrow, or maybe Monday.  Received: from USC-ECL by SU-AI with TCP/SMTP; 14 May 83 23:02:25 PDT Received: from MIT-ML by USC-ECL; Sat 14 May 83 22:59:14-PDT Received: from SCRC-EUPHRATES by SCRC-TENEX with CHAOS; Sun 15-May-83 01:05:34-EDT Date: Sunday, 15 May 1983, 01:56-EDT From: David A. Moon Subject: package changes To: DILL%CMU-CS-C@SU-DSN Cc: common-lisp@SU-AI In-reply-to: The message of 12 May 83 03:06-EDT from DILL at CMU-CS-C Date: 12 May 83 03:06:44 EDT From: DILL@CMU-CS-C .... I propose modifying Fahlman's proposal as below. 1. Flush INCLUDE-PACKAGE and UNINCLUDE-PACKAGE. The only reason I can see to have these functions is to build big packages out of little ones. Here is a more tasteful solution that requires no additional functions: a) Define a package to represent the whole system, and export from it all the symbols that users will want to import from it. b) For the little packages composing the system, build them in the natural way, except IMPORT all the symbols from the package of part a. Another way to group a bunch of packages would be to build another package which imports symbols from the grouped package, and then selectively exports them for importation by users. Or just have the little packages USE-PACKAGE the big package (this is how it works in the Lisp machine now, except that there is a spurious requirement of hierarchy rather than heterarchy). Or have the big package not export any symbols on its own, but have each of the little packages include an EXPORT statement with the big package as its second argument, taking their own symbols and exporting them as externals of the big package rather than as externals of the particular little package. If they want they could export symbols from both; it ought to work for a symbol to be an external of more than one package. In this case of course one needs to make a conscious decision what the home package of these symbols should be, which is really a conscious decision of what prefix one wants to see when the symbol is printed with a qualified name. You could want to see either the name of the big conglomerate package, or the name of the specific internal package that actually defines the symbol. An advantage of flushing INCLUDE-PACKAGE is that it makes the need for this decision more apparent. The gain in simplification of the implementation from flushing INCLUDE-PACKAGE is extremely small; I would guess between 15 and 20 lines of code. But I am in favor of this anyway, purely for documentation reasons. With the addition of a structured package name space of some kind, the names of the "internal" packages could also be isolated in order to avoid conflicts with user-level package names. We seem to have decided to avoid things like non-global package naming for now. It can easily be added later, but there are more user-interface issues than you would possibly believe beforehand, as we have found out through sad experience with the Lisp machine package system. 2. Flush DECLARE-PACKAGE. Require that ALL symbols exported by a package be exported before being imported (in any way) into other packages (modulo debugging). I never liked declare-package, and it does seem that the more we look at it the more problems occur. Unless Scott's next proposal manages to cleverly fix all the problems with declare-package, I am in favor of flushing it. I've pointed out before how in the simple cases of smallish programs, even including mutually-referencing packages, one can have the package-setup forms and the program-defining forms all in a single file while still conforming to the rule that a package must be set up before it is used. It only requires that LOAD be callable from inside LOAD (pace pdp10 Maclisp). Note: in the above I am using the ugly expression "set up" because in the Lisp machine this is called "declaring a package", while in the recent Common Lisp package discussion "declaring a package" has come to mean something else. I specifically mean "set up" to be what the author of the package does, with MAKE-PACKAGE or IN-PACKAGE or whatever it ends up being called. Don't call it SETUP-PACKAGE. 3. Define (USE-PACKAGE P) to IMPORT the symbols in package P's export list at the time that USE-PACKAGE is called ("copy" style). I pointed out in November why copy semantics is inherently unworkable for packages. But you may not have seen those comments. Mistakes in export lists will have to be remedied by tracking down the packages that have imported them. This may seem like a pain, Indeed it may. I think it is important for the package system to provide aid in tracking down such things. This is why reference rather than copy semantics is preferred. In fact, I'll add to my previous proposal, that to be workable SHADOW must record what it did in a structure slot in the package, that IMPORT must do the same thing. I also propose the following function: @defun[Fun {load-package} Args {@i[package-name]} @Optional {@i[file-name]}] If the package @i[package-name] exists, do nothing. Otherwise, load the file named by @i[file-name]. If this argument is not supplied, load the package from a file established by the system as the standard source for that package, signalling an error if there is no file established for @i[package-name]. If loading the file did not cause the package to exist, signal an error. USE-PACKAGE should call this function if the package it needs is not defined. Add an optional filename argument to USE-PACKAGE. Note that a load can be forced by calling LOAD explicitly. Essentially what you are proposing here is to conflate the ideas of "package" and "module" (laser manual page 291). The Lisp machine originally worked this way, but we abandoned it some years ago. I'm not sure that I can come up with an explanation of why this is not a good idea that would convince anyone not familiar with that history. Certainly in the simplest case modules (the units of separate maintenance, compilation, and loading) and packages (the units of separate symbol naming) are related one-to-one. But it is quite possible for a large module to involve more than one package, and it is also quite possible for a single package to contain more than one module. So it doesn't really make sense to replace these two distinct ideas with one single idea. Some Lisp machine background that may not be familiar to everyone on the list: We use the admittedly-overworked word "system" for what the laser manual calls a "module". This is because a system is a compound structure whose elements are called modules. A system is the unit of compilation, loading, and especially maintenance; thus a system is "a program." A module is the unit of dependency; thus one module may require loading a second module into the compiler, to define macros or whatever, before it may be compiled. Of course under both of these is a third unit, the file, which is the atomic unit of compilation and loading but is otherwise not intrinsically interesting. Systems can also be recursively constructed out of other systems. We have a general registry of "systems" and the files that must be loaded in order to load the system. Thus one would not normally supply a file name when demanding the loading of a system, but would simply let the general registry figure it out. There is a directory with one file for each system, the name of the file being derived in an obvious way from the name of the system. This file contains a Lisp form that specifies where the files of the system reside (sometimes using logical pathnames); this includes the file that contains the definition of the internal structure of the system, i.e. its inter-module dependencies and its package "set up". Other implementations would choose to do this in somewhat different ways. But Common Lisp can quite reasonably require every implementation to provide some primitive "library" mechanism of this type, so that one can call some function with a name like REQUIRE and the name of "a program", and be assured that that program is present in the Lisp environment. +++++++ Summary: Flush include-package. Keep use-package the way it is. Flush declare-package. Keep the provide/require stuff from the laser manual. Write documentation on how to use all this stuff, for both simple cases and complex cases.  Received: from CMU-CS-C by SU-AI with TCP/SMTP; 14 May 83 22:28:26 PDT Received: ID ; 15 May 83 01:28:02 EDT Date: Sun, 15 May 1983 01:28 EDT From: Scott E. Fahlman To: David A. Moon Cc: Common-Lisp@SU-AI, Kent M. Pitman Subject: New Error Proposal: suggestions to be incorporated In-reply-to: Msg of 15 May 1983 01:09-EDT from David A. Moon Dave, I see that you've bought some of KMP's suggestions. They look OK to me too. Several people in our group have independently suggested somehow getting a second format string into CERROR to say what happens if you proceed. I slightly prefer the ECASE format to the OTHERWISE-ERROR proposal, but I'm not adamant. Will you edit your error proposal to incorporate the new stuff? I'd like to have one message or file containing all the current stuff, rather than having to hunt around among N messages. Thanks, Scott  Received: from MIT-MC by SU-AI with TCP/SMTP; 14 May 83 22:07:51 PDT Received: from SCRC-EUPHRATES by SCRC-TENEX with CHAOS; Sun 15-May-83 00:18:40-EDT Date: Sunday, 15 May 1983, 01:09-EDT From: David A. Moon Subject: New Error Proposal: suggestions to be incorporated To: Kent M. Pitman Cc: Common-Lisp@SU-AI In-reply-to: The message of 11 May 83 05:59-EDT from Kent M. Pitman Date: 11 May 1983 05:59 EDT From: Kent M. Pitman * Based on heavy experience with the old and new error system, I feel strongly that one consistent irritation of the old system was that it didn't clearly distinguish between the cause and correction of an error and that CERROR as currently proposed will perpetuate that problem. eg, when you ran with error messages turned on but errors ignored, typeout like "continuing will do such-and-so" was at least irritating and frequently wrong. I put out for discussion the idea that the CERROR should be modified as follows... CERROR format-string-1 format-string-2 &rest args Signal an error, with the message constructed by applying FORMAT to format-string-1 and the arguments. If the debugger is entered, it may choose to offer a message composed by calling FORMAT on format-string-2 and the arguments, describing what will happen if the error is proceeded. Continuing from such an error will cause CERROR to return NIL. Use CERROR rather than ERROR to signal errors for which you have written recovery code. The name stands for "continuable error," which is too verbose to use for such a common function. Examples: (UNLESS (= (LIST-LENGTH FORM) 3) (CERROR "Wrong number of arguments in ~S" "~1G:~[Ignore extra args~;Assume 0 for missing args~]." FORM (< (LIST-LENGTH FORM) 3)) (SETQ FORM (APPEND FORM '(0 0)))) (DO () ((KNOWN-WORDP X) X) (CERROR "~S is unknown, probably misspelled." "Replace ~S and try again." X) (FORMAT T "~&New word: ") (SETQ X (READ))) You have an excellent point here. Consider this added to my error proposal. Let's amend the first example to avoid scaring people with hairy formatology: (UNLESS (= (LIST-LENGTH FORM) 3) (CERROR "Wrong number of arguments in ~S" (IF (< (LIST-LENGTH FORM) 3) "Assume 0 for missing args." "Ignore extra args.") FORM) (SETQ FORM (APPEND FORM '(0 0)))) Let's also include some kind of style suggestion on the second format string so that if a program automatically continues from the error it might be able to incorporate the string into an explanation of what it did. This may be overambitious since it won't work with any CERROR that asks for new input when continued, but it could still be worthwhile for a lot of things, and also may dovetail nicely into the condition system when we put one in some time in the future. * It occurs to me that the naming of CTYPECASE and ETYPECASE might be solved in a completely different way. Rather than adding new special forms, one could make several kinds of otherwise clauses. The argument against this might be that it means more special cases of symbols in the case-list position, making that position more visually ambiguous. The counter-argument (held by some (eg, me)) is that people should use parens around everything but the otherwise options anyway. The Common Lisp manual is written to encourage this style. Examples: (SETQ X 1/3) (TYPECASE X ((INTEGER) (- X)) ((SYMBOL) (INVERSE X)) (:OTHERWISE-ERROR)) Error: The value of X, 1/3, was neither an integer nor a symbol. (TYPECASE X ((INTEGER) (- X)) ((SYMBOL) (INVERSE X)) (:OTHERWISE-CERROR)) Since OTHERWISE is not a keyword, it would be too confusing for these to be keywords. They should be written without a colon. These are simply magic words that are part of the syntax of these special forms. I thought having to insert only a single letter to get this error checking might encourage people to do it more, since it would seem painless. However there is something to be said from a documentation point of view for your proposal and I'll go along with it if there is general sentiment in favor of it. I'm still mildly in favor of my original proposal for this, however. One thing I should point out is that the "ECASE" proposal can be generalized to COND if we decide we want that, but the "OTHERWISE-ERROR" proposal cannot be, except in a grotesque way. * I liked your stating explicitly not to put "Error: " at the head of error messages. I would further like it specified that multi line error messages should not be padded (with spaces, ";", etc.) because tools such as the debugger (or whatever) will take care of padding lines with the appropriate text. (ERROR (FORMAT NIL "FOO~%BAR")) should print out as: >>Error: FOO BAR not as >>Error: FOO BAR Ditto for the continuation string. The LispM currently has the bug that proceed options come out as: s-A: This is a proceed option extending over multiple lines. s-B: This is another. when s-A: This is a proceed option extending over multiple lines. s-B: This is another. would be better. Note that (ERROR (FORMAT NIL "FOO~% BAR")) would want to come out like >>Error: FOO BAR The reason for guaranteeing something like this is that users will frequently see the error coming out only in the debugger and may feel an urge to write (ERROR (FORMAT "FOO~%~16TBAR")) or some such so it'll look pretty. If another program gets ahold of such a format string and tries to use it elsewhere, though, it'll look like garbage. Better the user should just be told flat out that "any necessary padding for continuation lines will be taken care of". eg, I have seen Maclisp code where users did: (ERROR (FORMAT NIL "FOO~%;BAR")) since they knew that the first line would have a leading ";" and the rest would not. If the error message style is with leading, that should be supplied on line 1 and on continuation lines by the system. This also takes care of people who try to assume that line 1 will be shorter than subsequent lines because they know a 20 character leader will occur on that line and tends to produced multi-line messages where lines have balanced lengths, rather than multi-line messages where the first line is considerably shoerter than subsequent lines. I'd like to see all these stylistic suggestions go into the Common Lisp manual. * I don't see any reason why (ABORT-PROGRAM) can't be a synonym for what the LispM calls (SIGNAL 'SYS:ABORT). This is a pretty useful thing to put on interrupt characters. All you have to do is add an additional special form (CATCH-PROGRAM-ABORTS ...) or some such which does (CONDITION-CASE () (PROGN ...) (SYS:ABORT ...)) on the LispM. In a Lisp where conditions didn't exist, people could do the equivalent with CATCH/THROW. In fact, I don't know why just ABORT and CATCH-ABORT aren't adequate names. I find the word "program" ambiguous on a mainframe between meaning the toplevel form I invoked in my read-eval-print loop and the exec command I used to invoke the Lisp. (ie, the c-Abort vs c-m-Abort distinction on the LispM). In any case, I think these are valid kinds of issues for programmers to want to do in a machine independent fashion and should be included. The c-Abort vs c-m-Abort distinction is a real issue. CATCH-ABORT should properly include some sort of documentation of what it is that you are aborting from or to, as ERROR-RESTART in the Lisp machine does. I think it's going to be too hard to specify the semantics of aborting precisely enough to make it implementation-independent until we put condition-handling capabilities into Common Lisp. So let's just leave it out for now, so we don't define ourselves into a corner, even though we know very well that it is needed. * In things like CHECK-ARG, etc. I see no reason their forms shoudln't be evaluable multiple times. It's essential that it be clearly documented, but that's easy to do. In the rare case that multiple evaluation would be a screw, a simple LET will probably fix the user's problem. If we later find out it's easy to guarantee single evaluation, it's easier to relax things and tell him so than it is to go the other route if people have written code depending on single evaluation. Okay, I believe this.  Received: from CMU-CS-C by SU-AI with TCP/SMTP; 12 May 83 16:07:07 PDT Received: ID ; 12 May 83 19:07:13 EDT Date: Thu, 12 May 1983 19:07 EDT From: Scott E. Fahlman To: Daniel L. Weinreb Cc: common-lisp@su-ai Subject: More about packages, from DLW and BSG In-reply-to: Msg of 12 May 1983 16:48-EDT from Daniel L. Weinreb Gasp!..... . . Thud!  Received: from CMU-CS-C by SU-AI with TCP/SMTP; 12 May 83 16:10:13 PDT Received: ID ; 12 May 83 19:09:59 EDT Date: Thu, 12 May 1983 19:09 EDT From: Scott E. Fahlman To: Daniel L. Weinreb Cc: common-lisp@su-ai Subject: More about packages, from DLW and BSG In-reply-to: Msg of 12 May 1983 16:48-EDT from Daniel L. Weinreb Or in other words... Thanks for your extensive feedback, and I'll try to reflect all of those points in the next version of the proposal, which I didn't manage to finish last night, but will be finishing sometime in the next couple of days. Keep those cards and letters coming. -- Scott  Received: from MIT-MC by SU-AI with TCP/SMTP; 12 May 83 13:46:55 PDT Received: from SCRC-BORZOI by SCRC-TENEX with CHAOS; Thu 12-May-83 16:44:47-EDT Date: Thursday, 12 May 1983, 16:48-EDT From: Daniel L. Weinreb Subject: More about packages, from DLW and BSG To: common-lisp@su-ai Actual problems with the proposal: In your example, have you considered what would happen if you were to run the Lisp compiler on Newton's init file? As far as I can tell, the compiler would signal an error upon seeing the reference to the symbol "relativity:speed-of-light", because that package would not exist, at compile time. You need an eval-when around the load form. As the proposal is written, use-package will consider its arguments from left-to-right, and pushnew each one. This will effectively reverse the order of the list, which seems confusing. If you import an uninterned symbol, or otherwise cause a symbol to exist that is present in some package but has no home package, the rules for how to print the symbol (assume that package is the current package) both say that you should print it without any qualifier and that you should print it with a leading #:. Presumably the rule for how to print symbols should be amended to say that it only applies to interned symbols (those with a home package). The consistency rules are not violated, since they are careful to only talk about interned symbols. If an interned symbol is removed from its home package then the consistency rules will be violated (the symbol might be printed as foo:bar or foo#:bar even though it's not in that package). I don't know what to do about this. Places where the proposal is unclear: There is a fundamental question about the semantics of packages whose answer I was not sure of until I had read the proposal most carefully, namely, what does it mean for a symbol to be "external"? The real answer is that a given symbol can be interned in many packages, and in each of these packages it can either be internal or external. You can only ask whether a symbol is external in a certain package, not whether it is external. Let me assure you that this point is not at all obvious unless you read the proposal extremely carefully, and it caused us some confusion at various points. The first paragraph about external versus internal gives you the impression that it would be meaningful to have a function called externalp that just takes a symbol and returns a boolean value, by its use of language. An explicit definition of the algorithm for looking up an unqualified name was given, but no corresponding definition was given of the algorithm for looking up a name qualified with a colon. Suppose we are looking up the string "FOO:BAR", and FOO has an internal symbol named BAR, and FOO inherits from FIE, and FIE has an external symbol named BAR. Does it return the external BAR symbol in FIE, or does it signal an error because the symbol that it finds isn't external? It is not until the function find-external-symbol is defined that we are actually told the answer to this question (the former). I had initially assumed that "FOO:BAR" follows the standard lookup algorithm, and then signals an error if it finds that the symbol it has found is not an "external symbol". (Of course, there's no such thing as being an external symbol, but I still wasn't sure of that.) This point also needs more clarification earlier in the description, possibly by putting in a separate algorithm just to make things completely clear. (In particular, the wording "refers to the external symbol named @b[buffer] in the package named @b[editor]" is what threw me off most severely. I asked myself "what if @b[buffer] is an internal symbol?" and it wasn't clear what the answer was. The phrasing suffers from "present King of France" syndrome.) I am also unhappy with the sentence in the description of use-package that says "These symbols become available as @i[internal] symbols". This is non-strict phrasing, since the term "available" is only defined in such a way that you can ask whether a symbol is available or not; you can't ask whether it's available "as" an internal symbol. The sentence as phrased deserves to be followed by "so to speak"; it isn't literally meaningful, and gives me the impression that they "become internal symbols of the package", which they don't. The declare-package/error-checking issue: The statement that "no two symbols of the same name may be accessible from the same package" is trivially true, from the definition of "accessible"; after all, find-symbol can only return one symbol. What Moon is really saying here, I presume, is that no symbol should shadow any other symbol unless some explicit mechanism is used that says "hey, I'm really trying to shadow something". Moon said elsewhere that "the order of searching ancestors cannot be allowed to make a difference if we are to remain sane", which implies the same thing, more or less. I presume he's saying that every operation that might set up an implicit shadowing must be error-checked to make sure that it doesn't do so. This clearly means that the concept of "shadowed" must be added to the proposal and defined clearly. I would presume that shadowing, like external/internal, is a property of an "interning"; that is, a symbol can be interned in several packages, and in each package it might or might not be "shadowing". Someone must propose what functions exist to cause an interning to be shadowing, go through some examples of how you'd use shadowing, and figure out the algorithm for checking for name conflicts whenever use-package, inherit-package, export, or other functions are called given that some internings are shadowing and some are not. I think Dill's example (the P1, S1 one) shows that the algorithm is not immediately obvious, particularly when shadowing is involved. Until then it's hard to judge whether it all works or not. By the way, I also noticed the problem of "when do you know that you're finished loading?". My analysis of Moon's BLORPH example is that the situation that the various participants have asked for is that there should be one symbol, BLORPH, external in the A package, and packages B and C should be using this symbol. If BLORPH is created and exported in A before B and C are loaded, this happens, and otherwise an error gets signalled by Moon's proposed error-checking stuff. The existing Zetalisp package system deals with this by "locking" any package that is used by any other package; i.e. it never lets you add a new symbol into any ancestor package. In either case, Moon's scenario is not really a "screw case", since there is only one situation that can occur without an error being signalled. The official screw cases are where two different things can happen without any error being signalled. Moon's example shows how the current proposal can create this kind of screw. SEF's comment about "the old LispM package system, which sometimes does seem like it's out to screw people in all possible ways" is interesting in this light. In Dill's new proposal, the same problem is solved by using "copy semantics". If you export a symbol in P2 after P1 asked to use P2, then P1 just doesn't get the new symbol. So ancestors have to get set up before they're used. This is just like the Zetalisp philosophy, except that you don't get an error message if you do things out of order (because in the new definition, it's not an error). The order dependence is still there. In the Zetalisp package system, we did what we considered the simple and safe thing. We stated at the outset that every package must be declared before it is used, and the declaration sets up all relevant information about the package, saying what it inherits from, exports, shadows, etc. The package declaration must be evaluated (i.e. loaded) before anything else can even THINK about using the package. Enough rambling: the major decision to be made at this juncture is whether we want to try to let you use a package before its owner has declared it, or whether we want to make sure that the owner declares a package before anybody tries to use it. The present proposal attempts to do the former. One problem has been pointed out already (by Moon): declaring a package that is constructed by inclusion of other packages requires the the declarer know about this structure. I.e. if I want to use package foo, and package foo inherits from package bar, I have to know that. This is clearly unmodular. Here's something even worse about declaring a package that includes some other package. Consider (declare-package "foo" :include "bar :export '("a" "b" "c")) Are "a", "b", and "c" interned in "foo" or "bar"? The syntax of declare-package has to be extended to allow the caller to say so, explicitly. Are you grossed out yet? I think the current favorite counterproposal around here is to say that for every package, there is a file that sets up that package, specifying what it uses and includes and exports, and programs that want to access those packages would have something like Dill's load-package function to read in that file if it doesn't exist already. For small packages, the entire program that makes up that package could be in the file; for large packages, you have to get the rest of the files loaded yourself, but this one file has all the relevant forms that set up the package structure.  Received: from MIT-MC by SU-AI with TCP/SMTP; 12 May 83 12:00:01 PDT Date: Thursday, 12 May 1983 14:53-EDT From: MOON at SCRC-TENEX To: DILL at CMU-CS-C Cc: common-lisp at SU-AI Subject: package changes In-reply-to: The message of 12 May 83 03:06:44 EDT from DILL@CMU-CS-C The first part of your message is quite reasonable, except for the claim that it is very expensive to check for. When P2 exports S1', you simply find every package that uses or includes P2 and call (FIND-SYMBOL (SYMBOL-NAME S1') Pn) in that package. This is just INTERN that doesn't create new symbols. If this comes back with a symbol that isn't S1', you have a name conflict, unless the symbol it comes back with is on the shadow list of Pn. Of course, you don't ask the user questions one at a time; you find all the name conflicts first, sort them according to their various causes, then give the user one explanation of what happened. Continuing from the error would presumably be allowed with several options including both simple ones like "do it anyway" and "give up on exporting", and complex options like "ask me a separate question for each package with a conflict". I haven't studied the rest of your message yet. Will send more mail tomorrow or later today.  Received: from CMU-CS-C by SU-AI with TCP/SMTP; 12 May 83 03:21:34 PDT Received: ID ; 12 May 83 03:06:45 EDT Date: 12 May 83 03:06:44 EDT From: DILL@CMU-CS-C Subject: package changes To: common-lisp@SU-AI I'll give one more push for major changes before I decide that I'm obstructing progress. It is not clear to me how an implementation can reasonably assure that "no two symbols accessible to a package have the same name" with "reference style" USE-PACKAGE. Consider the following simple case of accidental shadowing: Pn is a package; Sm is a symbol. 1. P3 exports S1. 2. P1 uses P3 3. P1 uses P2. 4. P2 exports S1' (a non-EQ symbol with the same pname as S1). Read-Read consistency is violated, since the result of interning "S1" before step 4 will not be EQ to interning it after step 4. Similarly, Print-Read and Print-Print consistency are violated: try printing S1 immediately before step 4, Read the result or print it again after step 4. This problem can occur no matter what the searching order of the various packages is. It can be very expensive to check for. With INCLUDE-PACKAGE, you have to find all packages using P2, and check that there is either a symbol with pname "S1" appearing in the use list ahead of P2 or that there is no conflicting symbol appearing after P2 in the use list to be assured that this has not happened. With INCLUDE-PACKAGE, it is much more complicated, since step 4 may cause S1' to be exported to many different packages. A much worse problem is that after detecting this situation, the user will then have to deal with potentially a lot of questions demanding that he resolve conflicts between many symbols in many packages, perhaps for packages he knows nothing about. ---------------------------------------------------------------- I propose modifying Fahlman's proposal as below. 1. Flush INCLUDE-PACKAGE and UNINCLUDE-PACKAGE. The only reason I can see to have these functions is to build big packages out of little ones. Here is a more tasteful solution that requires no additional functions: a) Define a package to represent the whole system, and export from it all the symbols that users will want to import from it. b) For the little packages composing the system, build them in the natural way, except IMPORT all the symbols from the package of part a. These packages will define the various functions, properties, and values associated with these symbols. Each of these "internal" packages can hide its own internal symbols, and communicate with other internal packages by exporting symbols and importing them in the ordinary way. The user won't see any of these symbols unless he imports them intentionally from one of the internal packages. So, the user can see it as a single large system by just importing the package defined in part a. With the addition of a structured package name space of some kind, the names of the "internal" packages could also be isolated in order to avoid conflicts with user-level package names. Another way to group a bunch of packages would be to build another package which imports symbols from the grouped package, and then selectively exports them for importation by users. 2. Flush DECLARE-PACKAGE. Require that ALL symbols exported by a package be exported before being imported (in any way) into other packages (modulo debugging). 3. Define (USE-PACKAGE P) to IMPORT the symbols in package P's export list at the time that USE-PACKAGE is called ("copy" style). Mistakes in export lists will have to be remedied by tracking down the packages that have imported them. This may seem like a pain, but is probably inconsequential compared with the pain of tracking down wrong symbols that were read by a package importing from a bogus export list. The disadvantage of this definition that can be inferred from comments on proposals is that it won't be possible for packages to export new symbols from LISP and have them automatically appear in other packages. This is not a serious problem (I believe). No package author in his right mind would use the new LISP features without taking steps to ensure that they have been loaded -- and if he does this, he can equally well do a (USE-PACKAGE LISP), after doing so. The only potential exception is the USER package; for this, packages exporting from LISP can also explicitly IMPORT the symbols into USER. The combined effect of these changes greatly simplifies the explanation of how symbols are looked up, which is the closely related to the conceptual simplicity of the system. It is also safer. ---------------------------------------------------------------- I also propose the following function: @defun[Fun {load-package} Args {@i[package-name]} @Optional {@i[file-name]}] If the package @i[package-name] exists, do nothing. Otherwise, load the file named by @i[file-name]. If this argument is not supplied, load the package from a file established by the system as the standard source for that package, signalling an error if there is no file established for @i[package-name]. If loading the file did not cause the package to exist, signal an error. USE-PACKAGE should call this function if the package it needs is not defined. Add an optional filename argument to USE-PACKAGE. Note that a load can be forced by calling LOAD explicitly. ---------------------------------------------------------------- Here's how these changes would affect the example Fahlman gave in his proposal [my comments in square brackets]: [The init file would be simplified by not having to load the packages. USE-PACKAGE and LOAD-PACKAGE would be sufficient.] The file "alchemy.lisp": @begin [lisp] ;;; Alchemy functions, written and maintained by Merlin, Inc. ;;; This file uses some functions from "Phogiston.lisp". (declare-package "phlogiston" :export ("heat-flow" "cold-flow" "make-fire-bottle")) [delete this previous s-expression] ;;; All of the following is in the ALCHEMY package. (in-package "alchemy") ;;; Here is our external interface. (export '(lead-to-gold gold-to-lead vms-to-unix pawn-to-king4 elixir-of-life)) ;;; We use this a lot, so import it. [insert: (load-package phlogiston). The phlogiston package will be able to use our symbols, because we have already created the package and exported them. It will not try to load this package again because the package will already exist. Note that we have not used any symbols from phlogiston, yet, so it's okay to load it now, if necessary.] (import '(phlogiston:make-fire-bottle)) (defun lead-to-gold (x) "Takes a quantity of lead, returns gold." (when (> (phlogiston:heat-flow x) 3) ; Using a qualified symbol. (make-fire-bottle x)) ; Using an imported symbol. (gild x)) ;;; And so on ... @end [lisp] The file "phlogiston.lisp": @begin [lisp] ;;; Phlogiston functions, written and maintained by Thermofluidics, Ltd. ;;; This file uses the ALCHEMY package from file "alchemy.lisp". ;;; The symbols listed are the only ones actually referenced. (declare-package "alchemy" :export ("gold-to-lead" "elixir-of-life")) [delete this] ;;; All of the following is in the PHLOGISTON package. (in-package "phlogiston") ;;; Here is our external interface. (export '(heat-flow cold-flow mix-fluids separate-fluids burn make-fire-bottle)) ;;; We use this a lot, so import it. [same comments apply here as in the phlogiston. If we load this one first, it will not be loaded again, and it's symbols can be used by phlogiston] (use-package "alchemy") (defun heat-flow (amount x y) "Make some amount of heat flow from x to y." (when feeling-weak (quaff (elixir-of-life))) ; No qualifier needed. (push-heat amount x y)) ;;; And so on ... @end [lisp] [In Fahlman's example, he has an init file that loads both alchemy and phlogiston -- a call to HEAT-FLOW will not work unless both are loaded. In this proposal, the user can always (USE-PACKAGE "phlogiston") and call HEAT-FLOW. If it was pre-loaded in the init file, this will be faster, but in either case it will happen exactly once.] -Dave -------  Received: from MIT-MC by SU-AI with TCP/SMTP; 11 May 83 19:54:48 PDT Received: from SCRC-EUPHRATES by SCRC-TENEX with CHAOS; Wed 11-May-83 22:05:32-EDT Date: Wednesday, 11 May 1983, 22:09-EDT From: David A. Moon Subject: package.mss.108 To: Scott E. Fahlman Cc: Common-Lisp@su-ai In-reply-to: The message of 11 May 83 15:22-EDT from Scott E. Fahlman Date: Wed, 11 May 1983 15:22 EDT From: Scott E. Fahlman I'm still a little unclear on how the error-cheking in declare-package would be handled when a package is loaded from multiple files (as the LISP package would be, for instance). I guess the real issue is here is not so much multiple files as the setting up of a package's externals by a bunch of separate EXPORT "statements" rather than by a :EXPORT keyword to MAKE-PACKAGE (or whatever we call it), which is what I had assumed without thinking about it. Thus you don't really know when a package is all loaded so that you can error-check any declares that may have been done. To you, this means that the error checking is useless. To me, it means declare-package is a kludge. I'm not sure what we should do about this. If all else fails you could just put in a function that is called explicitly to do the error checking, and try to encourage people to put it at the end of their program or wherever. But note that in the example on p. 292 of the Laser edition of two programs that depend on each other, it appears that what you are expected to do is in this order: 1. Make your package 2. Do your export statements 3. Do your require statements, loading anything you need 4. Do your use statements, maybe adding what you just loaded to your package The key thing here is that the export statements are all done -before- anyone tries to look at the package from the outside, getting rid of the loading order issues. I guess it's up to you what style you want to promote. Since my style is to put all the package setup forms in separate files from the code, avoiding any issue of loading order dependency entirely, it doesn't matter to me.  Received: from USC-ECL by SU-AI with TCP/SMTP; 11 May 83 13:01:33 PDT Received: from MIT-ML by USC-ECL; Wed 11 May 83 12:59:08-PDT Date: Wednesday, 11 May 1983 15:37-EDT From: MOON at SCRC-TENEX To: common-lisp at su-ai Subject: need for export controls I was rather incoherent last night. The scenario where a problem occurs because a symbol is exported that shouldn't have been, perhaps because of a mistake in a declare-package, is as follows: Package A is use-package'd by packages B and C. The programs in B and C each use a symbol BLORPH, knowing it to be internal. Let's say they use it as an indicator to put on property lists. A gets a new external, BLORPH, that it didn't have before. Perhaps this is for real, or perhaps it's just a typo for BLEAGH. If this external is added after B and C have already been loaded and have each interned their own BLORPH, you get a name clash error (even if only one of B and C have been loaded). If this external is added when neither B nor C has been loaded yet, when they do get loaded and make their BLORPH, they both get the same symbol. If they can ever get "near each other," e.g. they both try to put BLORPH properties on the same symbol, you have an undetected name clash. Morals of the story: If you want to be very conservative, use SHADOW on all symbols that you use as unique, private identifiers, rather than simply making the (usually correct) assumption that you won't inherit anything by the name. There are two kinds of externals: the kind that are only accessed via qualified names, which are safe from undetected name clashes, and the kind that are accessed via use-package or include-package. Every new external of the latter kind must be accompanied by documentation when released to the world. As a third style (besides qualified names and wholesale use-package), B and C could have IMPORTed just the symbols they actually wanted from A. I don't think Common Lisp can legislate any of these three styles out of existence, though; consider the LISP package, which it would be silly to access in any but the "use" style. The name clash detection I am promoting will detect all clashes in the qualified name style (where it becomes ambiguous what a certain qualified name refers to), clashes in the import style where you accidentally referenced a name before you imported it and hence got an internal symbol, and clashes in the use style caused by modifying a package after someone has already inherited it. It cannot detect clashes in the import style where "the left hand doesn't know what the right hand is doing," i.e. two parts of a program in a single package don't agree about whether a certain name is to be private or imported. In general it cannot detect name clashes among the denizens of a single package! It cannot detect clashes in the use style where a new symbol, unanticipated by the inheriting programs, is exported by the ancestor program -before- the inheriting programs are loaded. This one I think is inherent in the nature of packages, or in the nature of contracts, and cannot be solved. It would be very nice if the Common Lisp manual, at least in the second edition, does not make the mistake we made in the Lisp machine manual and omit all discussion of these issues, on the grounds that it would scare the users or that it would make the manual too big. It's important to have a clear exposition of what the problems with packages are. Maybe I should just write (with help from others, surely) a freestanding paper on the subject.  Received: from CMU-CS-C by SU-AI with TCP/SMTP; 11 May 83 12:22:33 PDT Received: ID ; 11 May 83 15:22:28 EDT Date: Wed, 11 May 1983 15:22 EDT From: Scott E. Fahlman To: moon%scrc-tenex@mit-mc Cc: Common-Lisp@su-ai, fahlman@CMU-CS-C Subject: package.mss.108 Tried to send this out earlier, but lost because of incompatible mail-address protocols. Trying again... Dave, OK, given your clarifications, I guess I am almost completely happy with all the changes you propose. I'm still a little unclear on how the error-cheking in declare-package would be handled when a package is loaded from multiple files (as the LISP package would be, for instance), but I will review your note carefully and see if I can figure out something appropriate. I have a feeling that some of this emphasis on error checking is a slight over-reaction to your experience with the old LispM package system, which sometimes does seem like it's out to screw people in all possible ways. But given that Common Lisp will eventually be used by all sorts of non-sophisticated users who would be totally mystified by a package bug (now you see it, now you don't), I guess we'd better lean way over in the direction of safety. I yield to the voice of first-hand experience. While I'm on this theme, I withdraw my earlier suggestion that we flush &allow-other-keywords. Temporary insanity, the result of over-work. I will put all these changes in tonight, unless you think it would go quicker (fewer additional iterations) if you did the next edit. Let me know what you think will be easiest all around. If I don't hear from you by tonight, I'll do it. -- Scott  Received: from USC-ECL by SU-AI with TCP/SMTP; 11 May 83 13:01:33 PDT Received: from MIT-ML by USC-ECL; Wed 11 May 83 12:59:08-PDT Date: Wednesday, 11 May 1983 15:37-EDT From: MOON at SCRC-TENEX To: common-lisp at su-ai Subject: need for export controls I was rather incoherent last night. The scenario where a problem occurs because a symbol is exported that shouldn't have been, perhaps because of a mistake in a declare-package, is as follows: Package A is use-package'd by packages B and C. The programs in B and C each use a symbol BLORPH, knowing it to be internal. Let's say they use it as an indicator to put on property lists. A gets a new external, BLORPH, that it didn't have before. Perhaps this is for real, or perhaps it's just a typo for BLEAGH. If this external is added after B and C have already been loaded and have each interned their own BLORPH, you get a name clash error (even if only one of B and C have been loaded). If this external is added when neither B nor C has been loaded yet, when they do get loaded and make their BLORPH, they both get the same symbol. If they can ever get "near each other," e.g. they both try to put BLORPH properties on the same symbol, you have an undetected name clash. Morals of the story: If you want to be very conservative, use SHADOW on all symbols that you use as unique, private identifiers, rather than simply making the (usually correct) assumption that you won't inherit anything by the name. There are two kinds of externals: the kind that are only accessed via qualified names, which are safe from undetected name clashes, and the kind that are accessed via use-package or include-package. Every new external of the latter kind must be accompanied by documentation when released to the world. As a third style (besides qualified names and wholesale use-package), B and C could have IMPORTed just the symbols they actually wanted from A. I don't think Common Lisp can legislate any of these three styles out of existence, though; consider the LISP package, which it would be silly to access in any but the "use" style. The name clash detection I am promoting will detect all clashes in the qualified name style (where it becomes ambiguous what a certain qualified name refers to), clashes in the import style where you accidentally referenced a name before you imported it and hence got an internal symbol, and clashes in the use style caused by modifying a package after someone has already inherited it. It cannot detect clashes in the import style where "the left hand doesn't know what the right hand is doing," i.e. two parts of a program in a single package don't agree about whether a certain name is to be private or imported. In general it cannot detect name clashes among the denizens of a single package! It cannot detect clashes in the use style where a new symbol, unanticipated by the inheriting programs, is exported by the ancestor program -before- the inheriting programs are loaded. This one I think is inherent in the nature of packages, or in the nature of contracts, and cannot be solved. It would be very nice if the Common Lisp manual, at least in the second edition, does not make the mistake we made in the Lisp machine manual and omit all discussion of these issues, on the grounds that it would scare the users or that it would make the manual too big. It's important to have a clear exposition of what the problems with packages are. Maybe I should just write (with help from others, surely) a freestanding paper on the subject.  Received: from MIT-MC by SU-AI with TCP/SMTP; 11 May 83 03:09:32 PDT Date: 11 May 1983 05:59 EDT From: Kent M. Pitman Subject: New Error Proposal To: Moon @ SCRC-TENEX cc: Common-Lisp @ SU-AI The new error proposal looks mostly good. A few remarks, though... * Based on heavy experience with the old and new error system, I feel strongly that one consistent irritation of the old system was that it didn't clearly distinguish between the cause and correction of an error and that CERROR as currently proposed will perpetuate that problem. eg, when you ran with error messages turned on but errors ignored, typeout like "continuing will do such-and-so" was at least irritating and frequently wrong. I put out for discussion the idea that the CERROR should be modified as follows... CERROR format-string-1 format-string-2 &rest args Signal an error, with the message constructed by applying FORMAT to format-string-1 and the arguments. If the debugger is entered, it may choose to offer a message composed by calling FORMAT on format-string-2 and the arguments, describing what will happen if the error is proceeded. Continuing from such an error will cause CERROR to return NIL. Use CERROR rather than ERROR to signal errors for which you have written recovery code. The name stands for "continuable error," which is too verbose to use for such a common function. Examples: (UNLESS (= (LIST-LENGTH FORM) 3) (CERROR "Wrong number of arguments in ~S" "~1G:~[Ignore extra args~;Assume 0 for missing args~]." FORM (< (LIST-LENGTH FORM) 3)) (SETQ FORM (APPEND FORM '(0 0)))) (DO () ((KNOWN-WORDP X) X) (CERROR "~S is unknown, probably misspelled." "Replace ~S and try again." X) (FORMAT T "~&New word: ") (SETQ X (READ))) * It occurs to me that the naming of CTYPECASE and ETYPECASE might be solved in a completely different way. Rather than adding new special forms, one could make several kinds of otherwise clauses. The argument against this might be that it means more special cases of symbols in the case-list position, making that position more visually ambiguous. The counter-argument (held by some (eg, me)) is that people should use parens around everything but the otherwise options anyway. Examples: (SETQ X 1/3) (TYPECASE X ((INTEGER) (- X)) ((SYMBOL) (INVERSE X)) (:OTHERWISE-ERROR)) Error: The value of X, 1/3, was neither an integer nor a symbol. (TYPECASE X ((INTEGER) (- X)) ((SYMBOL) (INVERSE X)) (:OTHERWISE-CERROR)) Error: The value of X, 1/3, was neither an integer nor a symbol. Entering Debugger. Options: Meta-A: Store a new value in X and retry erring code. Meta-B: Return to Lisp toplevel. Meta-C: Logout >> Meta-A New Value for X: 9 -9 * I liked your stating explicitly not to put "Error: " at the head of error messages. I would further like it specified that multi line error messages should not be padded (with spaces, ";", etc.) because tools such as the debugger (or whatever) will take care of padding lines with the appropriate text. (ERROR (FORMAT NIL "FOO~%BAR")) should print out as: >>Error: FOO BAR not as >>Error: FOO BAR Ditto for the continuation string. The LispM currently has the bug that proceed options come out as: s-A: This is a proceed option extending over multiple lines. s-B: This is another. when s-A: This is a proceed option extending over multiple lines. s-B: This is another. would be better. Note that (ERROR (FORMAT NIL "FOO~% BAR")) would want to come out like >>Error: FOO BAR The reason for guaranteeing something like this is that users will frequently see the error coming out only in the debugger and may feel an urge to write (ERROR (FORMAT "FOO~%~16TBAR")) or some such so it'll look pretty. If another program gets ahold of such a format string and tries to use it elsewhere, though, it'll look like garbage. Better the user should just be told flat out that "any necessary padding for continuation lines will be taken care of". eg, I have seen Maclisp code where users did: (ERROR (FORMAT NIL "FOO~%;BAR")) since they knew that the first line would have a leading ";" and the rest would not. If the error message style is with leading, that should be supplied on line 1 and on continuation lines by the system. This also takes care of people who try to assume that line 1 will be shorter than subsequent lines because they know a 20 character leader will occur on that line and tends to produced multi-line messages where lines have balanced lengths, rather than multi-line messages where the first line is considerably shoerter than subsequent lines. * I don't see any reason why (ABORT-PROGRAM) can't be a synonym for what the LispM calls (SIGNAL 'SYS:ABORT). This is a pretty useful thing to put on interrupt characters. All you have to do is add an additional special form (CATCH-PROGRAM-ABORTS ...) or some such which does (CONDITION-CASE () (PROGN ...) (SYS:ABORT ...)) on the LispM. In a Lisp where conditions didn't exist, people could do the equivalent with CATCH/THROW. In fact, I don't know why just ABORT and CATCH-ABORT aren't adequate names. I find the word "program" ambiguous on a mainframe between meaning the toplevel form I invoked in my read-eval-print loop and the exec command I used to invoke the Lisp. (ie, the c-Abort vs c-m-Abort distinction on the LispM). In any case, I think these are valid kinds of issues for programmers to want to do in a machine independent fashion and should be included. * In things like CHECK-ARG, etc. I see no reason their forms shoudln't be evaluable multiple times. It's essential that it be clearly documented, but that's easy to do. In the rare case that multiple evaluation would be a screw, a simple LET will probably fix the user's problem. If we later find out it's easy to guarantee single evaluation, it's easier to relax things and tell him so than it is to go the other route if people have written code depending on single evaluation. --kmp  Received: from MIT-MC by SU-AI with TCP/SMTP; 10 May 83 23:50:02 PDT Date: 11 May 1983 02:48 EDT From: Kent M. Pitman Subject: Request for clarification about "package users/owners" To: Fahlman @ CMU-CS-C, Moon @ SCRC-TENEX cc: Common-Lisp @ SU-AI "... I don't like the idea of package A exporting symbols from package B; what if there is a typographical error, a conceptual mistake, or an obsolete symbol in the list supplied by package A, which is a duplicate of package B's list?..." "...I don't see how the "owner vs. user" error checking that you advocate can work unless you require that the exporting package be loaded before the using package (in which case the user doesn't need to declare the package at all)..." "What I object to ... is the notion that any of these operations, when performed in package X, have to worry about what's going on down in the descendant packages..." To paraphrase a popular slogan,... Packages don't export symbols, programs export symbols. Programs have meaning even (especially) after all its symbols have been seen and interned by READ, and possibly after the binding of *PACKAGE* has gone away. A program does not exist "in a package". The names of the entry points into a package are the only thing which are in the package. Anything else is simply code which sets up or implements some feature which allows that entry point does what it does, but properly speaking that code has nothing at all to do with packages. There is certainly no property about the code from a file which says "I am in the ... package." At best, packageness is a property of a written program's syntax. Not all programs, however, are written. Some, for example, are consed on the fly. Some, perhaps, are even compiled on the fly. I acknowledge Moon's concern about users vs owners of packages having various rights which may not be equivalent; I don't here want to take a stand on that issue. My real aim is to say that I am not sure I agree that it is trivially obvious what it means for one to be "in a package" or not in one, so the idea of error checking or otherwise controlling it seems exceedingly tricky/ill-defined. I hope my question is clear. It may well be that you mean something completely reasonable, but if so, I am not sure what that might be. If this message isn't clear, send me private mail and I'll clarify the question in greater length ... --kmp  Received: from MIT-MC by SU-AI with TCP/SMTP; 10 May 83 23:47:46 PDT Date: Wednesday, 11 May 1983 02:28-EDT From: MOON at SCRC-TENEX To: Scott E. Fahlman Cc: common-lisp at su-ai Subject: package.mss.108 In-reply-to: The message of Wed 11 May 1983 00:46 EDT from Scott E. Fahlman Date: Wed, 11 May 1983 00:46 EDT From: Scott E. Fahlman The use of a symbol argument (not a string) in FIND-ALL-SYMBOLS was deliberate. It didn't seem to me that interning a symbol (internally in the user package or wherever the user is typing) was something particularly to be avoided. With a string, you get all the issues of case, and with a symbol argument you automatically get the intuitive thing. But I've got no real problem with allowing either a string or a symbol here. Just in case you didn't catch it the first time around, my point here was that if you want to find out "are there any symbols named X, and if so where are they", asking the question shouldn't perturb the environment in a way that changes the answer, by creating a possibly-new symbol named X. In particular it's quite a useful response when FIND-ALL-SYMBOLS tells you "no, there aren't any symbols named X, anywhere". Let's just make it accept either a symbol or a string. Users calling it directly will pass symbols, probably, while user-interface tools will tend to pass strings. I'm not completely happy with declare-package and the :export keyword. I don't like the idea of package A exporting symbols from package B; what if there is a typographical error, a conceptual mistake, or an obsolete symbol in the list supplied by package A, which is a duplicate of package B's list? There is, of course, the problem of these lists getting out of synch, but I think that this problem is not likely to be a serious one, since the list in B is supposed to be a subset of the list in A -- B just declares those symbols from A that it actually uses.... I don't see how the "owner vs. user" error checking that you advocate can work unless you require that the exporting package be loaded before the using package (in which case the user doesn't need to declare the package at all). Let me be extra explicit here. "Loading" a package is what the program that "owns" that package does. "Declaring" a package is something anyone who uses it can do, if they don't require that the package's program be loaded before they are. A package may be declared any number of times, both before and after it has been loaded. A package contains a bit that is set when it is "loaded". Declaring a package after it has been loaded checks that the symbols exported by the declaration are a subset of the package's externals, native or inherited. Declaring a package before it exists creates the data structure but doesn't mark it loaded. Declaring a package before it is loaded creates externals according to the export list and does inheriting according to the include list, and doesn't do error checking. Loading a package that already exists, because it was declared one or more times before it was loaded, checks that the externals and inclusions the package has are subsets of those declared by the guy who is now loading it, who is the only one who knows what is really right. The rule that is implemented is that what the declarer says about the package must be a subset of what the loader says about it. This is checked as soon as they both have spoken. If the rule is violated, a continuable error is signalled and continuing acts as if the declarer knew what he was doing better than the loader. Note that declaring a package that is constructed by inclusion of other packages requires that the declarer, unmodularly, understand this structure. Maybe you don't use declaring much for that kind of package. [This paragraph has nothing to do with the subject at hand, and applies equally well to any package system, or any system that has globally named objects.] We arrange for something reasonable to happen if a package is loaded twice; the issue here is figuring out whether it was loaded by the same program, which was loaded twice for some reason, or whether two different programmers chose the same package name, unbeknownst to each other. This is probably done in an implementation-dependent way (the Lisp machine will base it on the name of the file containing the code that loaded the package). Some implementations will choose not to detect this kind of name conflict at all, while others will choose always to signal an error when a package is loaded twice, assuming that if the user is loading a program twice he knows about it and will continue from the error. (Previously I called this "making" a package, but that could be taken to mean the primitive that creates the package data structure.) In your writeup, name clashes are only discussed under IMPORT.... What I object to very strongly (I think) is the notion that any of these operations, when performed in package X, have to worry about what's going on down in the descendant packages. It's tricky enough to make this work looking up the tree, and I think it's impossible to sort it out looking down. All sorts of things might have been shadowed and moved around by explicit agreement, and trying to decide after the fact what is to be an error would be impossible. I don't know what direction is "up" for you and what direction is "down". But no matter which way I interpret it I don't believe it is impossible, or even hard. It is useless to do name clash checking if you don't do it completely. More below. My previous proposal said that every package contained a list of the symbols in that package that had been shadowed, which was specifically remembered for the use of name-clash error checking. Indeed if you don't remember what the user said to shadow you can't do name-clash checking. Once again, we are talking about adding a LOT of conceptual hair to handle a problem that will seldom occur and that will probably not screw anyone if it does occur. In practice, this is the problem that makes people frequently curse the day packages were ever invented. The point here is that while it may seem rare and unlikely to screw anyone to you, from your present abstract viewpoint, when a name clash occurs it is so unpredictable, and so pernicious in its effects, that it is very important to have iron clad assurance that these things can't happen. If I may make a strained analogy, suppose we came up with a really neat efficient garbage collection algorithm, based on hashing, which worked really well except that one in a million times, or say one in 2^32 times, it would throw away something it should have kept. Well, probably no one will ever notice, it's pretty rare, right? Incidentally I don't believe that this adds any conceptual hair whatsoever. It adds a small amount of implementation hair, in that you need backpointers from packages to their inheritors, but that is not conceptual hair. You already have the concept of shadowing. The rule "no two symbols with the same name may be accessible from the same package, unless one of them was explicitly declared via shadowing" seems the soul of simplicity to me. I'm then just proposing that the rule be enforced, i.e. it be "signals an error" rather than "is an error." Much better than the same rule, plus "except when include-package is used to more than one level" or "except when export is called after intern has already been called" or "except when debugging late at night and typing spastically."  Received: from CMU-CS-C by SU-AI with TCP/SMTP; 10 May 83 22:36:21 PDT Received: ID ; 11 May 83 00:58:57 EDT Date: Wed, 11 May 1983 00:58 EDT From: Scott E. Fahlman To: common-lisp@su-ai Subject: [Mailer: Message of 11 May 83 00:55:02 EDT] Date: 11 May 83 00:57:16 EDT From: The Mailer Daemon To: Re: Message of 11 May 83 00:55:02 EDT Message failed for the following: common-lisp@SU-AI: Timed out waiting for FTP reply ------------ Received: ID ; 11 May 83 00:55:02 EDT Date: Wed, 11 May 1983 00:55 EDT From: Scott E. Fahlman To: Alan Bawden Cc: common-lisp@SU-AI Subject: Two New Package Proposal Comments (and a grotesque suggestion) In-reply-to: Msg of 11 May 1983 00:20 EDT from Alan Bawden I'm not too fond of your grotesque suggestion. I just got done telling MOON that people are unlikely to have typos in the :export part of DECLARE-PACKAGE and you come up with this. Sigh! This string/case business is clearly a source of typos just waiting to happen, so something has to be done about that. Also, I guess if you call EXPORT, the system should see how long you've been awake and signal a correctable error if it's more than 20 or so hours. Only the clear-headed should be allowed to touch packages. -- Scott  Received: from CMU-CS-C by SU-AI with TCP/SMTP; 10 May 83 21:45:36 PDT Received: ID ; 11 May 83 00:46:19 EDT Date: Wed, 11 May 1983 00:46 EDT From: Scott E. Fahlman To: moon%scrc-tenex@mit-mc Cc: common-lisp@su-ai Subject: package.mss.108 Dave, I've got no problem with any of your "small" issues. I'll make all those changes. The use of a symbol argument (not a string) in FIND-ALL-SYMBOLS was deliberate. It didn't seem to me that interning a symbol (internally in the user package or wherever the user is typing) was something particularly to be avoided. With a string, you get all the issues of case, and with a symbol argument you automatically get the intuitive thing. But I've got no real problem with allowing either a string or a symbol here. On the two larger issues you raise, I think we have some fundamental disagreements. Medium: I'm not completely happy with declare-package and the :export keyword. I don't like the idea of package A exporting symbols from package B; what if there is a typographical error, a conceptual mistake, or an obsolete symbol in the list supplied by package A, which is a duplicate of package B's list? The symbols that are put into B by a DECLARE-PACKAGE statement in file A are declared in an export list, and you have to be careful when typing export lists. There is no reason to believe that B is going to be sloppier than A. There is, of course, the problem of these lists getting out of synch, but I think that this problem is not likely to be a serious one, since the list in B is supposed to be a subset of the list in A -- B just declares those symbols from A that it actually uses. If A gets augmented, that causes no problem for B. If something that B uses gets taken out of A (rare compared to augmentation), you're going to end up with an undefined function or unbound symbol bug, but that is unavoidable under any scheme. You also end up with a stray external symbol in package B because of the declaration in A, but the chances that this stray symbol is really going to screw anyone seem pretty small to me. So you have an error (getting out of synch) that COULD occur in a rare situation (removing an external symbol from some package's interface), and that is unlikely to cause trouble if it does occur. I don't see how the "owner vs. user" error checking that you advocate can work unless you require that the exporting package be loaded before the using package (in which case the user doesn't need to declare the package at all). In files that use external symbols from one another, all of the exports would have to be done before either file is loaded -- the package creation and exporting would have to be split into a separate file. Admittedly this is safer, but it is also a LOT less convenient. I just don't think that the danger of an occasional stray external, as described above, is worth that sort of restriction. Unless I'm overlooking some more common source of errors or you have some way to do the consistency checking without restricting load-order, I feel strongly that we should keep PACKAGE-DECLARE the way it is. Large: In your writeup, name clashes are only discussed under IMPORT. Name clashes should be given their own discussion section, and the following functions should be required to check for name clashes: EXPORT, IMPORT, USE-PACKAGE, INCLUDE-PACKAGE, and the implicit calls to those functions from MAKE-PACKAGE, DECLARE-PACKAGE, and IN-PACKAGE. When the externals of a package are altered, the check for name clashes must extend to all packages that inherit from that package (via either use or include). A name clash occurs when there is an attempt to make a symbol accessible in a package with the same name as a distinct symbol already accessible in that package, except when shadowing has been used to declare the user's intentions (there is a symbol already accessible in the package, with the same name, that was mentioned by SHADOW). When a name clash occurs, a correctable error is signalled. Well, there are two cases here. I have no strong objection to making IMPORT, USE-PACKAGE, and INCLUDE-PACKAGE look around when they are called and cause a correctable error if some symbol is about to be shadowed. We would have to add some sort of :INTENTIONALLY-SHADOWING list to these forms so that the error can be suppressed for specific known clashes, but that's OK. I also have no objection to having EXPORT look at its package's ancestors and signal an error if the newly-exported symbol would shadow an inherited one. What I object to very strongly (I think) is the notion that any of these operations, when performed in package X, have to worry about what's going on down in the descendant packages. It's tricky enough to make this work looking up the tree, and I think it's impossible to sort it out looking down. All sorts of things might have been shadowed and moved around by explicit agreement, and trying to decide after the fact what is to be an error would be impossible. Once again, we are talking about adding a LOT of conceptual hair to handle a problem that will seldom occur and that will probably not screw anyone if it does occur. It will very rarely be the case that you want to mess with the external symbols of a package after other packages have begun using it, and even if you do, it is unlikely to cause any trouble. If (EXPORT FOO) is called in package X and package Y, which uses X, has no FOO of its own, there's no problem. If Y does have a FOO, it will continue to use its own FOO, and things are no worse than before. If Y includes X (relatively rare), it is just the same, except that Y's descendants now see X:FOO, while Y sees its own internal FOO. But the descendants couldn't have wanted to get Y's internal FOO, so this is OK. I'm sure there's some way to construct a bug out of all this, but it doesn't look to me like it's going to be a problem for real users. -- Scott  Received: from MIT-MC by SU-AI with TCP/SMTP; 10 May 83 21:18:26 PDT Date: 11 May 1983 00:20 EDT From: Alan Bawden Subject: Two New Package Proposal Comments (and a grotesque suggestion) To: Fahlman @ CMU-CS-C cc: common-lisp @ SU-AI In-reply-to: Msg of Tue 10 May 1983 02:51 EDT from Scott E. Fahlman In Newton's init file it says: (use-package calculus). I presume this is a typo for (use-package "calculus") or perhaps (use-package 'calculus). I also presume (declare-package "phlogiston" :export ("heat-flow" "cold-flow" "make-fire-bottle")) should read (declare-package "phlogiston" :export ("HEAT-FLOW" "COLD-FLOW" "MAKE-FIRE-BOTTLE")) Since the case of these various string arguments will become the case of the exported symbols. It sure does SHOUT at you though. If we allowed symbols to be treated as strings here, then (declare-package "phlogiston" :export (:heat-flow :cold-flow :make-fire-bottle)) will work, although it is a bit misleading. Perhaps (declare-package "phlogiston" :export (#:heat-flow #:cold-flow #:make-fire-bottle)) ???  Received: from MIT-MC by SU-AI with TCP/SMTP; 10 May 83 20:39:36 PDT Date: 10 May 1983 23:41 EDT From: Alan Bawden Subject: Summary of message on function objects & :allow-other-keys To: BENSON @ SPA-NIMBUS cc: Common-Lisp @ SU-AI In-reply-to: Msg of 10 May 1983 17:12-PDT from BENSON at SPA-NIMBUS :allow-other-keys sounds ok to me. Date: Tuesday, 10 May 1983, 17:12-PDT From: BENSON at SPA-NIMBUS The first argument to APPLY or FUNCALL (for which FUNCTIONP is true) must be a compiled-code object or a lexical closure. Anything else is an error. I would object to this. I certainly agree that there is an inelegance in allowing symbols to serve as functions when passed as arguments to APPLY and FUNCALL (Algol 68 would be proud of a feature like this), but this is not an inelegance with any far-reaching consequences (assuming we clarify the manual about the term "closure", which it does tend to use rather randomly), and it is a traditional convenience supported by most of Common Lisp's ancestors. I don't see any compelling reason to introduce an ENCLOSE function, or to make environments first-class data objects. The latter seems like a unnecessary constraint on implementors. (I might go for an ENCLOSE like in the Scheme "Revised Report" that takes a \representation/ of an environment, but I still don't see the need for it.) A function to retrieve the lambda-expression from a closure also seems to me to be assuming a bit to much about the internals of a closure (why should that information still be there? What happens if it is a \compiled/ closure). I presume each implementation can provide rational debugging facilities to allow me to determine just where a closure originated.  Received: from CMU-CS-C by SU-AI with TCP/SMTP; 10 May 83 20:12:21 PDT Received: ID ; 10 May 83 23:10:00 EDT Date: Tue, 10 May 1983 23:09 EDT From: Scott E. Fahlman To: MOON%SCRC-TENEX@mit-mc, Kent M. Pitman Cc: Common-Lisp@SU-AI Subject: Packages In-reply-to: Msg of 10 May 1983 22:19-EDT from MOON at SCRC-TENEX I essentially agree with everything MOON said in response to KMP's message. I was about to send a very similar message myself. The only thing I would add is that INIT files are nowhere defined in the white pages (and probably should not be), but if they were, there is no reason that they have to follow all the rules LOAD does. Certain kinds of state, like the value of *PACKAGE*, want to be preserved across a LOAD, while an INIT file may well want to change assorted aspects of the global state. So I don't see a problem here. I do have some problems with a couple of the things MOON suggested in his earlier message on packages, and I'm working on a response to that. -- Scott  Received: from MIT-MC by SU-AI with TCP/SMTP; 10 May 83 19:40:57 PDT Date: Tuesday, 10 May 1983 22:19-EDT From: MOON at SCRC-TENEX To: Kent M. Pitman Cc: Common-Lisp at SU-AI Subject: Packages In-reply-to: The message of 10 May 1983 18:09 EDT from Kent M. Pitman Date: 10 May 1983 18:09 EDT From: Kent M. Pitman I'd like to respond to this at length since I think I understand all this stuff now. (1) I dislike "#:" in the middle of a name for several reasons. * I think it is ugly. It visually breaks up the symbol in a bad way. This is precisely why I suggested it. The internal-symbol qualified name syntax is very rarely used, and if it is used in a program the programmer is almost certainly doing something wrong, probably due to a misunderstanding. So I thought it was a good idea for it to be ugly. * "#" is defined as a token terminator. This would have to be redefined. I note (p219, Colander Edition -- sorry; I have no Laser edition) It's unfortunate that you weren't sent a Laser edition; this was fixed at the discussion meeting in Pittsburg last August. * I would prefer a prefix notation such as "#:FOO:BAR" rather than "FOO#:BAR". (If not "#:", some other prefix dispatch.) Thinking ahead to the multi-level packages for a moment, and given that package ids are strings (not symbols), would one write "FOO#:BAR:BAZ" or "FOO#:BAR#:BAZ" or ... I would prefer they lead off with an identification that the symbol coming up will be an internal one and then give a normal qualified reference. eg, you could define #: to just bind some (unexported) variable *COMPLAIN-ABOUT-INTERNAL-PACKAGE-REFERNCES* to NIL and re-call READ without hairing up the language syntax. I can't make any sense of this. #: is no more and no less hairing up the language syntax than : itself is. * If "#:" is used to mark symbols with no package cell, it is overloaded and visually misleading to have it mean both things. I don't think so; certainly no more than dot is overloaded. Before a symbol and inside of a symbol are just different, that's all. (2) I dislike visually marking symbols that have no package cell. This was agreed upon in August. Again it's unfortunate that you never saw a Laser edition manual. But if we go back and argue over everything we've done in the past year we'll never get this language defined. Doubly unfortunate that you never saw the quotation from the Federalist papers in the front of the laser edition. As it happens I don't much like marking uninterned symbols specially myself, but I'm not going to quarrel over it. Easy enough to put in a *prinfoo* flag to turn it off it really irritates. * When writing symbols out to a file and reading them back in, there's a high percentage of chance it really doesn't matter what package they are read into. To take a simple example, (READ-FROM-STRING (FORMAT NIL "~S" (LET ((V (GENSYM))) `(LAMBDA (,V) ,V)))) won't even "work". This just means that you're one of the large community of users who will always run with *princircle* set to t, commendably preferring correctness over efficiency. This doesn't have much to do with packages, or even symbols, though, really. You'd expect this with gensyms but its more irritating if you've inherited some symbol from another package and you've uninterned the symbol from the source package. Suddenly, perfectly valid symbols may appear as if "gensymed" and will lose this property of being EQ. Since inheriting is "by reference" rather than "by value", this can't happen. The inherited symbol isn't accessible if it gets uninterned from the place it is being inherited from. This is important. But perhaps you really meant to say "imported." You are right about this misfeature, which was mentioned in Fahlman's documentation. Removing the flagging of uninterned symbols would hardly fix this misfeature, it would simply obscure it somewhat, since now you would have "perfectly valid symbols" that would print without a package prefix when they should have had a prefix. The only advice I can give is if you move a symbol from one package to another and forget to setf the symbol-package to the new package, then you forgot something. I guess the following properties are relevant ... (a) Symbols which were EQ before a PRINT-READ cycle should be EQ afterward. (b) Symbols which were not EQ before a PRINT-READ cycle should not be EQ afterward. (c) A symbol which was EQ to some other symbol not involved in a PRINT-READ cycle should still be EQ to it after such a cycle. Given that property (c) is in general very hard to assure, I can't think of any examples of cases where it is at all difficult to assure. Can you supply one? I still place a higher priority on (a) than on (b). Printing symbols out with no annotation will guarantee (a) but may violate (b). Printing symbols out with the proposed "#:" marking (as I understand it) seems to violate (a) while guaranteeing (b). Again, *princircle* is the answer to this. Of course it can only work within a single form, so it isn't a complete answer, which by the very definition of interning is impossible. Further, I think people will see "#:" coming out before gensysms for ages before they see the other case. I predict this will cause great debugging confusion for people that don't understand the other cases where this can happen and who have from many experiences come to associate "#:" with gensyminess, not with the heuristic feature off of which it drives. Maybe. That depends on whether prefixes and infixes are considered to be the same by most people. (3) I'm not sure about using PUSHNEW rather than PUSH after a DELETE when calling things like USE-PACKAGE. It seems to me that users will expect the most recent call to USE-PACKAGE to have priority. Imagine a package A with an external symbol FOO on it. Imagine a package B which does USE-PACKAGE A and then shadows FOO. Imagine some package C, the writer of which notices that A:FOO isn't provided in B and who doesn't realize that A is included in B. He might ask to USE-PACKAGE A and be confused that A:FOO still isn't visible. Correcting your typo of USE-PACKAGE when you meant INCLUDE-PACKAGE, this scenario is exactly why I pointed our in my comments the need for more comprehensive handling of name clashes. The order of searching ancestors cannot be allowed to make a difference if we are to remain sane. (4) "If @b[unexport] is given a symbol that is already available as an internal symbol in the current package, it does nothing; ..." I guess my view was that "external" symbols are in some sense a subset of "internal" ones; ie, when working "internally" to a package, i expect "external" symbols are available (and they are). I was surprised, then, by this wording. Does it mean the same as, and would it be clearer as: "If @b[unexport] is given a symbol that is already available only as an internal symbol in the current package, it does nothing; ..." The wording probably needs to be clarified. I think Fahlman's intention was that the terms "internal" and "external" were to be taken as mutually exclusive. Internal is synonymous to neither "accessible" nor "present". (5) In DECLARE-PACKAGE's description , shouldn't you say @b[:nicknames], etc. or @i[nicknames]. I should think @i[:nicknames] would be confusing. (6) Re: LOAD binding *PACKAGE* The LispM has a problem with init files. People like to call pkg-goto and assign BASE,IBASE in their inits and find themselves thwarted by implicitly bound variables. A popular way to circumvent this problem is by doing (PROCESS-RUN-FUNCTION #'(LAMBDA () (SETQ IBASE 10.)), etc. This seems pretty drastic. Will Common Lisp address this issue? In the Lisp machine, this was recently corrected by introducing a new special form (macro), SETQ-GLOBALLY, along with accompanying functions SET-GLOBALLY, SYMEVAL-GLOBALLY, MAKUNBOUND-GLOBALLY. These operate on the global value of a dynamic variable, regardless of any bindings that may currently be in effect. They are slow. SETQ-GLOBALLY is also undoable. I hesitate to propose these for Common Lisp right now while we are in a rush to get a consistent, stable, documented language that we can all implement and use as a basis for future standardized extensions. (7) Are packages always global, named objects? Will there be a provision for anonymous packages or packages rooted elsewhere besides GLOBAL? In such an event, should LIST-ALL-PACKAGES be constrained to return even anonymous packages? Fahlman wanted to keep the proposal simple so yes there is only one namespace of package names and you aren't allowed to call the primitive that creates a package object without giving it a name and linking it into the data structure. Packages are non-hierarchical so "rooted" makes no sense. Are anonymous packages good for something? I expect that on the Lisp machine there will be a provision for nicknames to a package that are only recognized in qualified names and only when *package* is one of certain other packages. This extension to Common Lisp is good when renaming a whole group of related packages. This may be what you want. (8) IMPORT, EXPORT, etc. use the "current package". I would prefer to have them take &OPTIONAL (PACKAGE *PACKAGE*) since I find it very ugly to write (LET ((PACKAGE *PACKAGE*)) (IMPORT ...)) if I happen not to have that package selected. I agree. (9) There are occasionally times when one would like a symbol to be printed with any package prefix unconditionally. Such is the case for APROPOS, for example. I would like the method by which I can make this happen be explicit. On the LispM, one resorts to binding PACKAGE to NIL or (pkg-find-package "GLOBAL"), but these "solutions" lose awfully if you get a breakpoint in those packages since you either end up finding you have a bad value for package (and recursively losing) or not finding that you do (and losing in other ways). Perhaps a variable *FORCE-PACKAGE-PREFIXES* which could be set T/NIL. This would be a good name but unfortunately the name that is consistent with the other printer control flags is *PRINPACKAGE*. I proposed this at one point and would still like to see it; after we agree on the basic package system I will send a message containing a small list of non-controversial extensions like this which I would like to see included in the standard language. Better still, perhaps a variable *PRINT-PACKAGE-PREFIXES* which could be set :ALWAYS Always print package prefixes :NEVER Never print package prefixes :MAYBE Print package prefixes when not "obvious" fn A one-arg predicate which returns T if the package prefix should be displayed and NIL otherwise. Having more choices than T and NIL seems like a good idea here, although I'm not sure that the full hair of a predicate would be used. Setting it to :NEVER would be great when you want to do simple symbol manipulation unrelated to Lisp variables without getting bogged down in package I/O behavior. I have some applications where this would have been very handy. I don't understand this. What problem is solved by changing it from the default state (:MAYBE ?) to :NEVER? No symbol you can type in without a package prefix will be printed back out with one.  Received: from CMU-CS-C by SU-AI with TCP/SMTP; 10 May 83 18:40:43 PDT Received: ID ; 10 May 83 21:41:15 EDT Date: Tue, 10 May 1983 21:41 EDT From: Scott E. Fahlman To: DILL@CMU-CS-C Cc: common-lisp@SU-AI Subject: package proposal comments In-reply-to: Msg of 10 May 83 01:14:21 EDT from DILL A quick response to your comments (which reached me to late to be reflected in the latest package proposal): Transitive and Intransitive inheritance: The latest proposal provides the user with a choice. Copy vs. Pointer semantics: Both systems have problems. I now believe that copy semantics has worse problems, but I could be wrong. In any event, I'm reluctant to completely redo the proposal the other way unless the problems with deep-binding inheritance are fatal. Packages as the unit of loading and compilation: We've been around this a few times, and always come up with lots of problems. Loading the right set of files is a tricky and implementation-dependent business. I would like to see a comprehensive version-maintenance system in Common Lisp someday (need a thesis topic?) but there is no way we're going to get all that ironed out in time to make the first edition of the manual. -- Scott  Received: from CMU-CS-C by SU-AI with TCP/SMTP; 10 May 83 18:07:30 PDT Received: ID ; 10 May 83 21:08:29 EDT Date: Tue, 10 May 1983 21:08 EDT From: Scott E. Fahlman To: MOON%SCRC-TENEX @ mit-mc Cc: common-lisp@SU-AI Subject: &allow-other-keys No, I don't think that missing required args should default to nil or that extraneous arguments should be ignored. But keyword lists have a sort of built-in safety about them that counted-and-ordered argument lists don't have. Or at least they would if misspelling could be ruled out. I don't really expect to win on this one, and am not entirely sure that I want to. A trial balloon, as they say in Washington. If we continue with the current &allow-other-keys business, the :allow-other-keys extension doesn't seem too bad. It adds complexity, but I have nothing less complex to propose. -- Scott  Received: from MIT-MC by SU-AI with TCP/SMTP; 10 May 83 17:28:39 PDT Date: Tuesday, 10 May 1983 20:26-EDT From: MOON at SCRC-TENEX To: Scott E. Fahlman Cc: common-lisp at SU-AI Subject: not about suggestion for ballot: :allow-other-keys In-reply-to: The message of Tue 10 May 1983 18:57 EDT from Scott E. Fahlman Date: Tue, 10 May 1983 18:57 EDT From: Scott E. Fahlman In-reply-to: Msg of 10 May 1983 15:26-EDT from David A. Moon I've always thought that &allow-other-keys was a crock, and that unrecognized keywords should just be quietly ignored. Does this mean you also think that missing required arguments should default to nil and extraneous arguments should be quietly ignored? Note that I said that supplying an unrecognized keyword "is an error", not "signals an error." This was quite deliberate. I also never said whether the checking was done at run time, link time, or compile time, also quite deliberately. The rest of your message speaks for itself.  Received: from USC-ECL by SU-AI with TCP/SMTP; 10 May 83 17:19:41 PDT Received: from MIT-XX by USC-ECL; Tue 10 May 83 17:15:22-PDT Date: Tuesday, 10 May 1983, 17:12-PDT From: BENSON at SPA-NIMBUS Subject: Summary of message on function objects To: BENSON at SPA-NIMBUS, Common-Lisp%su-ai at usc-ecl In-reply-to: The message of 10 May 83 13:11-PDT from BENSON at SPA-NIMBUS To summarize the important points of the previous note: The Common Lisp manual must make clear the distinction between function objects (lexical closures and compiled-code objects) and function names (lambda expressions and symbols), and the evaluation process which produces function objects from function names. The first argument to APPLY or FUNCALL (for which FUNCTIONP is true) must be a compiled-code object or a lexical closure. Anything else is an error. Lexical environments are Lisp objects which need a section in the data types chapter. ENCLOSE takes a lambda expression and a lexical environment and returns a lexical closure. CLOSURE-EXPRESSION takes a lexical closure and returns the lambda expression which it encloses. CLOSURE-ENVIRONMENT takes a lexical closure and returns the lexical environment it was closed over.  Received: from USC-ECL by SU-AI with TCP/SMTP; 10 May 83 16:35:19 PDT Received: from MIT-MC by USC-ECL; Tue 10 May 83 13:26:59-PDT Date: Tuesday, 10 May 1983, 13:11-PDT From: BENSON at SPA-NIMBUS Subject: Function objects To: Common-Lisp%su-ai at usc-ecl The Laser edition is inconsistent in its treatment of functions. This inconsistency is due to confusion between the name of a function and the actual function object. This confusion has been around for a long time, and is manifested in many Lisp implementations where the name of a function object and the object itself are the same. To quote from the Standard Lisp Report: "FUNCTION is like QUOTE but its argument may be affected by compilation." First, we need to state what we mean by a function object. The definition of the FUNCTIONP data type predicate says that it "is true if its argument is suitable for applying to arguments, using for example the FUNCALL or APPLY function." The definition of APPLY states that its argument "may be a compiled-code object, or a lambda expression, or a symbol; in the latter case the global functional value of that symbol is used." (Let's ignore compiled-code objects for the time being; we'll get back to them later.) This is the definition which confuses the name (symbol or lambda expression) and the function object. Conflicting with this definition is the Functions section of the Progam Structure chapter (Section 5.2, p. 42). It states that "Lambda-expressions and symbols as names of functions can appear only as the first element of a function-call form, or as the second element of the FUNCTION special form." The definition of the FUNCTION special form, where FN is the second element of the form, states that "if FN is a symbol, the functional value of the variable whose name is that symbol is returned. If FN is a lambda expression, then a lexical closure is returned." Notice that the definition of APPLY doesn't even mention lexical closures, yet these are the things which are produced by the FUNCTION special form, which is presumably the preferred means of obtaining a suitable object to use as an argument to APPLY! The key to understanding all of this is to realize that the CAR of a Lisp form (ignoring macros and special forms) is not "not evaluated," in fact it is a function name which is evaluated to produce a function object. In Scheme this evaluation is the same as that used to produce function arguments, i.e. the function EVAL. Common Lisp must use a different algorithm, because it permits symbols to have distinct variable and function values. The function which implements this algorithm is not mentioned in the Common Lisp manual. In fact I don't know of any Lisp implementations which provide it; it is usually open-coded (sometimes a no-op!) within EVAL. For the sake of discussion, let's call it FEVAL. In fact the one we are most interested in is *FEVAL, analogous to *EVAL, since this function must use the current lexical environment in its evaluation process. Evaluation of a form which represents an ordinary function call is thus: FEVAL the CAR of the form, EVAL all the other elements of the list, call the result of FEVAL with the EVALed arguments. FEVAL is useful to produce a function object which will not be called immediately. The FUNCTION special form says in effect: use FEVAL instead of EVAL to produce this argument. Conversely, EVAL is also useful in producing function objects. FUNCALL allows normal argument evaluation to return a function object to be called. Note that neither FUNCTION nor FUNCALL is needed in Scheme precisely because EVAL and FEVAL are the same. FEVAL is a very simple function. If its argument is a lambda expression, it returns a closure of the expression over the lexical environment. If it is a symbol, the "functional variable" value of the symbol in the lexical (or global) environment is returned. Anything else is an error. As an upward-compatible extension, function specs as provided in Zetalisp, such as (:PROPERTY FOO BAR), are treated essentially the same as symbols. They simply specify a place from which to fetch a function object. The semantics of compiled-code objects have not been described extensively in the Common Lisp manual. They are mentioned in the spurious definition of APPLY above, which might lead one to think of them as being "like lambda expressions, only faster." In fact the only sensible interpretation of a compiled-code object is "like FEVAL of a lambda expression, only faster." In other words, anything a lexical closure can do, a compiled-code object can do. This should be stated explicitly in the manual in the data type section. The argument to APPLY (and FUNCALL) can be a compiled-code object or a lexical closure; anything else is an error. A language standard which considers symbols or lambda expressions to be functions is perpetuating the confusion described above. To acheive the effect of using a symbol as a function as described above (and currently implemented in Zetalisp, for example), that is, using the global functional value at the time it is called, one must use a closure which calls the function, e.g. to give BAR as a functional argument to FOO, where BAR's definition may change between the call to FOO and the FUNCALL of its functional argument, use: (FOO #'(LAMBDA (X) (BAR X))) ;BAR is a function of one argument instead of (FOO 'BAR) In general, if the number of arguments is unknown: (FOO #'(LAMBDA (&REST X) (APPLY #'BAR X))) Or, if you really mean the global function value, not the lexical: (FOO #'(LAMBDA (&REST X) (APPLY (SYMBOL-FUNCTION 'BAR) X))) The current language definition does not provide either a function which takes a lambda expression and a lexical environment and returns a lexical closure (*FEVAL) or functions to extract these components from an existing lexical closure. These aren't strictly necessary, but without them many useful parts of a programming environment must be written non-portably. For example, GRINDEF is impossible in portable Common Lisp, since SYMBOL-FUNCTION must return either a lexical closure or a compiled-code object. A structure editor which can redefine a function requires both selector functions and the constructor function, since it must create a new closure with the modified function body. I recommend that lexical environments be included in the language as first-class objects. *EVAL and EVALHOOK would then be defined to take exactly two arguments. ENCLOSE (*FEVAL restricted to lambda expressions) takes a lambda expression and a lexical environment and returns a lexical closure. CLOSURE-EXPRESSION and CLOSURE-ENVIRONMENT take a lexical closure and return its components. The constant NULL-ENVIRONMENT, the empty lexical environment, should be defined. The structure of a lexical environment need not (indeed, should not) be specified. These additions are not major and would aid immensely in supporting a truly lexical Lisp.  Received: from CMU-CS-C by SU-AI with TCP/SMTP; 10 May 83 15:56:35 PDT Received: ID ; 10 May 83 18:57:12 EDT Date: Tue, 10 May 1983 18:57 EDT From: Scott E. Fahlman To: David A. Moon Cc: common-lisp@SU-AI Subject: suggestion for ballot: :allow-other-keys In-reply-to: Msg of 10 May 1983 15:26-EDT from David A. Moon Urk! I've always thought that &allow-other-keys was a crock, and that unrecognized keywords should just be quietly ignored. That would solve this problem in a truly elegant way and speed up calls to keyword functions by a moderate factor, since it takes a separate scan of the argument list to look for bogus keywords. I will admit that signalling the unrecognized-keyword error is useful in catching spelling errors, but a good editor top-level could take care of that. Just thought I'd rasie this possibility before we add yet another epicycle. -- Scott  Received: from MIT-MC by SU-AI with TCP/SMTP; 10 May 83 15:08:00 PDT Date: 10 May 1983 18:09 EDT From: Kent M. Pitman Subject: Packages To: Common-Lisp @ SU-AI Here are my notes on CMUC:PACKAGE.MSS.108. Summary Item Importance Notes (1) Medium Propose changing "FOO#:BAR" to "#:FOO:BAR". (2) Large Propose flushing "#:" notation on symbols with NIL in their package cell. (3) Small Questioning use of PUSHNEW in USE-PACKAGE. (4) Small Wording issue in definition of "UNEXPORT". (5) Tiny Syntax in "DECLARE-PACKAGE". (6) Medium Setting *PACKAGE*, etc in init files. (7) Small LIST-ALL-PACKAGES and anonymous packages. (8) Medium Adding an optional PACKAGE argument to IMPORT, etc. (9) Medium Forcing symbols to print with[out] package prefixes. ----- (1) I dislike "#:" in the middle of a name for several reasons. * I think it is ugly. It visually breaks up the symbol in a bad way. * "#" is defined as a token terminator. This would have to be redefined. I note (p219, Colander Edition -- sorry; I have no Laser edition) there are no non-terminating macro sequences in the attributes table. I think this is wise from the point of view of novices. The presence of initially defined non-terminating macro sequences would just further complicate the knowledge of Lisp syntax a novice would need. * I would prefer a prefix notation such as "#:FOO:BAR" rather than "FOO#:BAR". (If not "#:", some other prefix dispatch.) Thinking ahead to the multi-level packages for a moment, and given that package ids are strings (not symbols), would one write "FOO#:BAR:BAZ" or "FOO#:BAR#:BAZ" or ... I would prefer they lead off with an identification that the symbol coming up will be an internal one and then give a normal qualified reference. eg, you could define #: to just bind some (unexported) variable *COMPLAIN-ABOUT-INTERNAL-PACKAGE-REFERNCES* to NIL and re-call READ without hairing up the language syntax. * If "#:" is used to mark symbols with no package cell, it is overloaded and visually misleading to have it mean both things. (2) I dislike visually marking symbols that have no package cell. * When writing symbols out to a file and reading them back in, there's a high percentage of chance it really doesn't matter what package they are read into. To take a simple example, (READ-FROM-STRING (FORMAT NIL "~S" (LET ((V (GENSYM))) `(LAMBDA (,V) ,V)))) won't even "work". You'd expect this with gensyms but its more irritating if you've inherited some symbol from another package and you've uninterned the symbol from the source package. Suddenly, perfectly valid symbols may appear as if "gensymed" and will lose this property of being EQ. I guess the following properties are relevant ... (a) Symbols which were EQ before a PRINT-READ cycle should be EQ afterward. (b) Symbols which were not EQ before a PRINT-READ cycle should not be EQ afterward. (c) A symbol which was EQ to some other symbol not involved in a PRINT-READ cycle should still be EQ to it after such a cycle. Given that property (c) is in general very hard to assure, I still place a higher priority on (a) than on (b). Printing symbols out with no annotation will guarantee (a) but may violate (b). Printing symbols out with the proposed "#:" marking (as I understand it) seems to violate (a) while guaranteeing (b). Further, I think people will see "#:" coming out before gensysms for ages before they see the other case. I predict this will cause great debugging confusion for people that don't understand the other cases where this can happen and who have from many experiences come to associate "#:" with gensyminess, not with the heuristic feature off of which it drives. An organized plan for treating gensyms would be nice, but in the absence of one (I don't think we have time for that now) I don't think that marking things that appear to be gensyms. As a minor issue, "#:" would then be free for use in item (1) above. (3) I'm not sure about using PUSHNEW rather than PUSH after a DELETE when calling things like USE-PACKAGE. It seems to me that users will expect the most recent call to USE-PACKAGE to have priority. Imagine a package A with an external symbol FOO on it. Imagine a package B which does USE-PACKAGE A and then shadows FOO. Imagine some package C, the writer of which notices that A:FOO isn't provided in B and who doesn't realize that A is included in B. He might ask to USE-PACKAGE A and be confused that A:FOO still isn't visible. I'm not so much saying this should be changed as I'm curious why things are supposed to happen as documented. Is there an alternative scenario that makes mine look less important? (4) "If @b[unexport] is given a symbol that is already available as an internal symbol in the current package, it does nothing; ..." I guess my view was that "external" symbols are in some sense a subset of "internal" ones; ie, when working "internally" to a package, i expect "external" symbols are available (and they are). I was surprised, then, by this wording. Does it mean the same as, and would it be clearer as: "If @b[unexport] is given a symbol that is already available only as an internal symbol in the current package, it does nothing; ..." (5) In DECLARE-PACKAGE's description , shouldn't you say @b[:nicknames], etc. or @i[nicknames]. I should think @i[:nicknames] would be confusing. (6) Re: LOAD binding *PACKAGE* The LispM has a problem with init files. People like to call pkg-goto and assign BASE,IBASE in their inits and find themselves thwarted by implicitly bound variables. A popular way to circumvent this problem is by doing (PROCESS-RUN-FUNCTION #'(LAMBDA () (SETQ IBASE 10.)), etc. This seems pretty drastic. Will Common Lisp address this issue? (7) Are packages always global, named objects? Will there be a provision for anonymous packages or packages rooted elsewhere besides GLOBAL? In such an event, should LIST-ALL-PACKAGES be constrained to return even anonymous packages? (8) IMPORT, EXPORT, etc. use the "current package". I would prefer to have them take &OPTIONAL (PACKAGE *PACKAGE*) since I find it very ugly to write (LET ((PACKAGE *PACKAGE*)) (IMPORT ...)) if I happen not to have that package selected. (9) There are occasionally times when one would like a symbol to be printed with any package prefix unconditionally. Such is the case for APROPOS, for example. I would like the method by which I can make this happen be explicit. On the LispM, one resorts to binding PACKAGE to NIL or (pkg-find-package "GLOBAL"), but these "solutions" lose awfully if you get a breakpoint in those packages since you either end up finding you have a bad value for package (and recursively losing) or not finding that you do (and losing in other ways). Perhaps a variable *FORCE-PACKAGE-PREFIXES* which could be set T/NIL. Better still, perhaps a variable *PRINT-PACKAGE-PREFIXES* which could be set :ALWAYS Always print package prefixes :NEVER Never print package prefixes :MAYBE Print package prefixes when not "obvious" fn A one-arg predicate which returns T if the package prefix should be displayed and NIL otherwise. Setting it to :NEVER would be great when you want to do simple symbol manipulation unrelated to Lisp variables without getting bogged down in package I/O behavior. I have some applications where this would have been very handy. --kmp  Received: from USC-ECL by SU-AI with TCP/SMTP; 10 May 83 14:51:38 PDT Received: from MIT-ML by USC-ECL; Tue 10 May 83 14:20:28-PDT Date: Tuesday, 10 May 1983, 14:19-PDT From: BENSON at SPA-NIMBUS Subject: suggestion for ballot: :allow-other-keys To: common-lisp at SU-AI In-reply-to: The message of 10 May 83 12:26-PDT from David A. Moon In fact they are mirror images of each other: &ALLOW-OTHER-KEYS is the receiver's means of permitting other keywords (which may or may not be captured in a &REST argument), and :ALLOW-OTHER-KEYS T is the transmitter's means of permitting other keywords. At first glance it seems to be a kludge, but any other user-level facility for this functionality will require something similar in its implementation, i.e. a parameter to give to the keyword parsing code.  Received: from USC-ECL by SU-AI with TCP/SMTP; 10 May 83 14:35:11 PDT Received: from MIT-MC by USC-ECL; Tue 10 May 83 13:26:59-PDT Date: Tuesday, 10 May 1983, 13:11-PDT From: BENSON at SPA-NIMBUS Subject: Function objects To: Common-Lisp%su-ai at usc-ecl The Laser edition is inconsistent in its treatment of functions. This inconsistency is due to confusion between the name of a function and the actual function object. This confusion has been around for a long time, and is manifested in many Lisp implementations where the name of a function object and the object itself are the same. To quote from the Standard Lisp Report: "FUNCTION is like QUOTE but its argument may be affected by compilation." First, we need to state what we mean by a function object. The definition of the FUNCTIONP data type predicate says that it "is true if its argument is suitable for applying to arguments, using for example the FUNCALL or APPLY function." The definition of APPLY states that its argument "may be a compiled-code object, or a lambda expression, or a symbol; in the latter case the global functional value of that symbol is used." (Let's ignore compiled-code objects for the time being; we'll get back to them later.) This is the definition which confuses the name (symbol or lambda expression) and the function object. Conflicting with this definition is the Functions section of the Progam Structure chapter (Section 5.2, p. 42). It states that "Lambda-expressions and symbols as names of functions can appear only as the first element of a function-call form, or as the second element of the FUNCTION special form." The definition of the FUNCTION special form, where FN is the second element of the form, states that "if FN is a symbol, the functional value of the variable whose name is that symbol is returned. If FN is a lambda expression, then a lexical closure is returned." Notice that the definition of APPLY doesn't even mention lexical closures, yet these are the things which are produced by the FUNCTION special form, which is presumably the preferred means of obtaining a suitable object to use as an argument to APPLY! The key to understanding all of this is to realize that the CAR of a Lisp form (ignoring macros and special forms) is not "not evaluated," in fact it is a function name which is evaluated to produce a function object. In Scheme this evaluation is the same as that used to produce function arguments, i.e. the function EVAL. Common Lisp must use a different algorithm, because it permits symbols to have distinct variable and function values. The function which implements this algorithm is not mentioned in the Common Lisp manual. In fact I don't know of any Lisp implementations which provide it; it is usually open-coded (sometimes a no-op!) within EVAL. For the sake of discussion, let's call it FEVAL. In fact the one we are most interested in is *FEVAL, analogous to *EVAL, since this function must use the current lexical environment in its evaluation process. Evaluation of a form which represents an ordinary function call is thus: FEVAL the CAR of the form, EVAL all the other elements of the list, call the result of FEVAL with the EVALed arguments. FEVAL is useful to produce a function object which will not be called immediately. The FUNCTION special form says in effect: use FEVAL instead of EVAL to produce this argument. Conversely, EVAL is also useful in producing function objects. FUNCALL allows normal argument evaluation to return a function object to be called. Note that neither FUNCTION nor FUNCALL is needed in Scheme precisely because EVAL and FEVAL are the same. FEVAL is a very simple function. If its argument is a lambda expression, it returns a closure of the expression over the lexical environment. If it is a symbol, the "functional variable" value of the symbol in the lexical (or global) environment is returned. Anything else is an error. As an upward-compatible extension, function specs as provided in Zetalisp, such as (:PROPERTY FOO BAR), are treated essentially the same as symbols. They simply specify a place from which to fetch a function object. The semantics of compiled-code objects have not been described extensively in the Common Lisp manual. They are mentioned in the spurious definition of APPLY above, which might lead one to think of them as being "like lambda expressions, only faster." In fact the only sensible interpretation of a compiled-code object is "like FEVAL of a lambda expression, only faster." In other words, anything a lexical closure can do, a compiled-code object can do. This should be stated explicitly in the manual in the data type section. The argument to APPLY (and FUNCALL) can be a compiled-code object or a lexical closure; anything else is an error. A language standard which considers symbols or lambda expressions to be functions is perpetuating the confusion described above. To acheive the effect of using a symbol as a function as described above (and currently implemented in Zetalisp, for example), that is, using the global functional value at the time it is called, one must use a closure which calls the function, e.g. to give BAR as a functional argument to FOO, where BAR's definition may change between the call to FOO and the FUNCALL of its functional argument, use: (FOO #'(LAMBDA (X) (BAR X))) ;BAR is a function of one argument instead of (FOO 'BAR) In general, if the number of arguments is unknown: (FOO #'(LAMBDA (&REST X) (APPLY #'BAR X))) Or, if you really mean the global function value, not the lexical: (FOO #'(LAMBDA (&REST X) (APPLY (SYMBOL-FUNCTION 'BAR) X))) The current language definition does not provide either a function which takes a lambda expression and a lexical environment and returns a lexical closure (*FEVAL) or functions to extract these components from an existing lexical closure. These aren't strictly necessary, but without them many useful parts of a programming environment must be written non-portably. For example, GRINDEF is impossible in portable Common Lisp, since SYMBOL-FUNCTION must return either a lexical closure or a compiled-code object. A structure editor which can redefine a function requires both selector functions and the constructor function, since it must create a new closure with the modified function body. I recommend that lexical environments be included in the language as first-class objects. *EVAL and EVALHOOK would then be defined to take exactly two arguments. ENCLOSE (*FEVAL restricted to lambda expressions) takes a lambda expression and a lexical environment and returns a lexical closure. CLOSURE-EXPRESSION and CLOSURE-ENVIRONMENT take a lexical closure and return its components. The constant NULL-ENVIRONMENT, the empty lexical environment, should be defined. The structure of a lexical environment need not (indeed, should not) be specified. These additions are not major and would aid immensely in supporting a truly lexical Lisp.  Received: from MIT-MC by SU-AI with TCP/SMTP; 10 May 83 12:28:51 PDT Date: 10 May 1983 1526-EDT From: David A. Moon Subject: suggestion for ballot: :allow-other-keys To: common-lisp@SU-AI This is a suggestion to address the query on page 71 of the laser edition. I'm able to figure out whether this a kludge or not, but I'd like to solicit a ballot on whether it is too inelegant to include in the language. Please read the manual's writeups on &key before voting unless you understand keyword arguments already. In addition to the keyword arguments explicitly declared, all functions with &key in their lambda list accept an additional keyword, :allow-other-keys. If this keyword is supplied and the value is true, then no error is signalled if the arguments to the function include unrecognized keywords. If :allow-other-keys is unsupplied or supplied with an argument of nil, and the &allow-other-keys symbol is not used in the lambda list, then it is an error if the argument list contains keywords not listed in the lambda list. This allows one to construct a keyword list that has been checked at some high level, secure in the knowledge that no error will be signalled at a low level. The keyword list can be passed to several different functions, each of which will extract the keywords that are relevant to it. Note that both &allow-other-keys and :allow-other-keys are useful features--neither subsumes the other. -------  Received: from MIT-MC by SU-AI with TCP/SMTP; 10 May 83 09:55:52 PDT Date: Tuesday, 10 May 1983 12:53-EDT From: MOON at SCRC-TENEX To: Fahlman at cmu-cs-c cc: common-lisp at su-ai, Moon at SCRC-TENEX Subject: package.mss.108 I am almost completely happy with this. There is one big problem, one medium-sized problem, and a few small problems. Small: Does the list returned by package-nicknames include the primary name? Say explicitly. find-all-symbols has to accept a string as an argument so that one doesn't spuriously create a symbol just to use as an argument. I suppose one could use an uninterned symbol, but it seems better to me to make it accept either a string or a symbol, or else just a string. The string of course has to be in the same case as a symbol's print-name; case-dependent lookup is used. There are several typographical errors in the description of do-symbols. Add do-external-symbols EXPORT and related functions should take an optional second argument which is the package to operate on, or its name, defaulting to *PACKAGE*. This is mainly valuable when you have a module that presents multiple interfaces and hence has some "extra" packages associated with it; the external symbols of each package correspond to one interface to that module. The code is all loaded into one package, but different things are exported to different packages; some symbols are present in more than one interface and hence exported to more than one package. Medium: I'm not completely happy with declare-package and the :export keyword. I don't like the idea of package A exporting symbols from package B; what if there is a typographical error, a conceptual mistake, or an obsolete symbol in the list supplied by package A, which is a duplicate of package B's list? I'd like to see some error-checking here. I suggest that there be some distinction between declare-package done by the "owner" of the package vs. declare-package done by some "user" of the package, and a consistency check between them (what is exported by a user is a subset of what is exported by the owner, and likewise for what is include-package'd). Possibly declare-package should be the "user" function, make-package should be the "owner" function, and make-package should err if the package has been made already but not if it has been declared already. in-package properly should be a variant of make-package, not of declare-package, in this scenario. However it is desirable to be able to go "into" a package more than once; I suggest that perhaps it is wrong for in-package to do an implicit make, and the make should be done separately. Large: The statement "No two symbols in the same package may have the same name" is not enlarged upon, and furthermore since this doesn't use the accessible/present terminology that was just defined, it isn't clear which is meant. To me, it is clear that the package consistency rules require that no two symbols -accessible- from the same package have the same name, unless one has been explicitly shadowed. Allowing implicit shadowing makes the result (which of the two symbols is actually accessible) dependent upon the order in which the symbols were first seen, or on the order that use-package and include-package put things onto lists, and destroys read-read consistency. In your writeup, name clashes are only discussed under IMPORT. Name clashes should be given their own discussion section, and the following functions should be required to check for name clashes: EXPORT, IMPORT, USE-PACKAGE, INCLUDE-PACKAGE, and the implicit calls to those functions from MAKE-PACKAGE, DECLARE-PACKAGE, and IN-PACKAGE. When the externals of a package are altered, the check for name clashes must extend to all packages that inherit from that package (via either use or include). A name clash occurs when there is an attempt to make a symbol accessible in a package with the same name as a distinct symbol already accessible in that package, except when shadowing has been used to declare the user's intentions (there is a symbol already accessible in the package, with the same name, that was mentioned by SHADOW). When a name clash occurs, a correctable error is signalled. A variety of ways to continue from the error may be provided by an implementation, including such possibilities as declining to perform the invalid operation, uninterning or unexporting the other symbol, automatically shadowing, and (in some implementations) linking function and value cells to make the two clashing symbols equivalent for those uses of symbols (but not equivalent for EQ of course) and uninterning one of them. It may also prove desirable in practice to have variables that can be set to cause automatic continuation from name clashes; but we shouldn't standardize on those yet.  Received: from MIT-MC by SU-AI with TCP/SMTP; 10 May 83 08:34:01 PDT Date: Tuesday, 10 May 1983 11:34-EDT From: MOON at SCRC-TENEX To: DILL at CMU-CS-C Cc: common-lisp at SU-AI Subject: package proposal comments In-reply-to: The message of 10 May 83 01:14:21 EDT from DILL@CMU-CS-C I think you'll find that the package system that finally gets adopted is consistent with your comments.  Received: from CMU-CS-C by SU-AI with TCP/SMTP; 9 May 83 23:51:14 PDT Received: ID ; 10 May 83 02:51:38 EDT Date: Tue, 10 May 1983 02:51 EDT From: Scott E. Fahlman To: common-lisp@su-ai Subject: New Package Proposal I can't seem to get my new and improved package proposal over the net to SAIL, maybe because it is too long. Anyway, people who want to see it should FTP it from CMUC. The file is package.mss. Share and Enjoy, Scott  Received: from SAIL by SCORE with Pup; Mon 9 May 83 22:19:44-PDT Received: from CMU-CS-C by SU-AI with TCP/SMTP; 9 May 83 22:14:41 PDT Received: ID ; 10 May 83 01:14:21 EDT Date: 10 May 83 01:14:21 EDT From: DILL@CMU-CS-C Subject: package proposal comments To: common-lisp@SU-AI Here are some thoughts about alternatives in the design of the CL package system, after looking over the original Fahlman proposal and the comments by Symbolics. I've tried to separate out considerations that are mostly independent, and tried to make explicit any assumptions about how packages are to be used. I won't present a lot of detailed alternatives, because Scott has probably seen them before in previous proposals. TRANSITIVE VS. INTRANSITIVE "INHERITANCE": As a general principle, users should be able to ask for only the symbols (definitions, whatever) they need from a package. This reduces the chances of an accidental name conflict among symbols that the user didn't need anyway. If transitive inheritance is the primary way for package A to gain access to package B's definitions, it violates this principle, because it automatically re-exports all sorts of symbols that may have nothing to do with the facilities that package A wants to provide. For a really huge system (e.g. MACSYMA or the LISP system itself), the implementors may want to think of the system as being made up of a bunch of reasonable-sized packages, while users may want to think of it as a single package (MACSYMA or LISP). For this purpose, it may be useful for a package to group together symbols from other packages and export them in a bunch. It is not clear that you need more than one level of "inheritance" for this, though. By the way, there seems to be a reasonably clean way to do multiple interfaces to the same package if it is possible to re-export a symbol that has been imported from somewhere else. Export the union of the symbols for all of the interfaces from the package that does the actual definitions, then import these into a separate new package for each interface. Then each of these new packages can export only those symbols that are relevant to the particular interface it is associated with. If this is really worth doing... "COPY" VS. "POINTER" SEMANTICS FOR IMPORTED SYMBOLS There have been two general types of package proposals within the Spice Lisp group in the last two years: when package A uses package B's symbols, does it get the symbols exported by package B at one particular time ("copy"), or does it have some sort of pointer to B, so that it can track changes in B's exported symbols over time ("pointer")? "Pointer" semantics seem to require an implicit or explicit order for searching export lists. When it is desirable to track changes, pointer semantics is sometimes more convenient. The two cases (that I can think of) where changes in export lists might happen are when someone is debugging a package and wants to correct a mistake in an export list, and when someone has decided to export a new symbol from an old package (only really legitimate when exporting from the LISP package, I think, if then). This does not necessarily work as well as it ought to. Usually, symbols will have been read and interned using one configuration of export lists, and when the export lists change, there will still be no easy way of correctly updating the old symbols to be what they should have been. The most serious disadvantage of the pointer semantics is that accidental shadowing is inherent when you attempt to track changes in export lists. Suppose package A searches (in order) the export lists of packages B and C. If a symbol is added to the export list of package B, it could shadow a symbols in C that A was previously using. Since there are a lot of packages that could be searching B and C in different orders (and a lot of different values for B and C), this situation is very difficult to check for and correct. With copy semantics, a package always knows what symbols it has gotten from another package immediately after it gets them. If a conflict arises, it can be resolved at that time, and no shadowing can occur without explicit participation by the packages affected. If symbols are added to an export list, it is relatively easy to ask for all the new symbols (but you still have to ask). It may be more difficult to forget symbols that have been removed from an export list, since it may be difficult to tell where they were imported from (not necessarily the home package, due to re-exporting). The current proposal seems to use "copy" semantics for IMPORTing an individual symbol, and "pointer" semantics for using all the symbols exported by another package: IMPORTed symbols will not be tracked if they are removed from an export list. In pure "copy" semantics, getting the export list of another package is equivalent to individually importing all those symbols. Some of these opinions are based on experience I have had using the MUDDLE package system, which is basically a "pointer" system with intransitive inheritance. In my experience, changing an export list almost never "worked" except when exporting from the MUDDLE equivalent of the LISP package. PACKAGES AS A UNIT OF LOADING If it is assumed that packages correspond closely to a set of definitions that should be loaded together, we could provide a function to load a package, given only the package name, only if it has not already been loaded. The system can find the appropriate file to load from, since the user probably doesn't care as long as the package gets loaded. Loading only if not already loaded is a win because many different packages may import from the same package -- it would be a pain to have to check in each case whether to load it. This also makes mutually referential packages (package A imports from package B which imports from A) easy. IMPORTing from a package could cause it to be loaded if necessary, as above. MACLISP style autoloading is not necessarily a substitute for this capability; if there are "autoload" properties on symbols, the packages defining those symbols will have to be defined to the extent of having created the package object and storing the relevant symbols somewhere. PACKAGES AS A UNIT OF COMPILATION A package system that clearly separates out the external and internal symbols can make it possible for a compiler to do a better job, since it can assume that internal definitions will not be accessed outside of the package. There are a lot of "block compilation" kinds of optimizations that could be done. This is not something that is generally done in the Maclisp/Lisp Machine world now, but I think we ought to leave open the possibility of doing it in some implementations in the future. Two ways to avoid losing this opportunity are to have a warning in the manual that you should only use internal symbols from another package if you absolutely know what you're doing, and having a different form of qualification for internal symbols (I favor "::" but basically don't care). I think I can generate a list of interesting optimizations if anyone is interested. -------