Changes

<br/>The normal way to deal with a bibliography is to refer to entries using a unique tag or key. When a list of entries is typeset, this reference can be used for linking purposes. The typeset list can be processed and sorted using the <tt style="color:rgb(0,102,102);font-size:100%;" >bibtex</tt> program that converts the database into something more <span tag="MKIV" style="font-style:sans;">TEX</span> friendly (a <tt style="color:rgb(0,102,102);font-size:100%;" >.bbl</tt> file). I never used the program myself (nor bibliographies) so I will not go into too much detail here, if only because all I say can be wrong. <br/>In <span tag="MKIV" style="font-style:sans;">ConTEXt</span> we no longer use the <tt style="color:rgb(0,102,102);font-size:100%;" >bibtex</tt> program: we just use database files and deal with the necessary manipulations directly in <span tag="MKIV" style="font-style:sans;">ConTEXt</span>. One or more such databases can be used and combined with additional entries defined within the document. We can have several such datasets active at the same time. <br/>A <span tag="MKIV" style="font-style:sans;">bibTEX</span> file looks like this:

<br/>Normally a value is given between quotes (or curly brackets) but single words are also OK (there is no real benefit in not using quotes, so we advise to always use them). There can be many more fields and instead of strings one can use predefined shortcuts. The title for example quite often contains <span tag="MKIV" style="font-style:sans;">TEX</span> macros. Some fields, like <tt style="color:rgb(0,102,102);font-size:100%;" >pages</tt> have funny characters such as the endash (typically as <tt style="color:rgb(0,102,102);font-size:100%;" >--</tt>) so we have a mixture of data and typesetting directives. If you are covering non--english references, you often need characters that are not in the <span tag="MKIV" style="font-style:sans;">ascii</span> subset but <span tag="MKIV" style="font-style:sans;">ConTEXt</span> is quite happy with <span tag="MKIV" style="font-style:sans;">utf</span>. If your database file uses old-fashioned <span tag="MKIV" style="font-style:sans;">TEX</span> accent commands then these will be internally converted automatically to <span tag="MKIV" style="font-style:sans;">utf</span>. Commands (macros) are converted to an indirect call, which is quite robust. <br/>The <span tag="MKIV" style="font-style:sans;">bibTEX</span> files are loaded in memory as <span tag="MKIV" style="font-style:sans;">Lua</span> table but can be converted to <span tag="MKIV" style="font-style:sans;">xml</span> so that we can access them in a more flexible way, but that is a subject for specialists. <br/>In the old <span tag="MKIV" style="font-style:sans;">MkII</span> setup we have two kinds of entries: the ones that come from the <span tag="MKIV" style="font-style:sans;">bibTEX</span> run and user supplied ones. We no longer rely on <span tag="MKIV" style="font-style:sans;">bibTEX</span> output but we do still support the user supplied definitions. These were in fact prepared in a way that suits the processing of <span tag="MKIV" style="font-style:sans;">bibTEX</span> generated entries. The next variant reflects the <span tag="MKIV" style="font-style:sans;">ConTEXt</span> recoding of the old <span tag="MKIV" style="font-style:sans;">bibTEX</span> output.

<br/>The split <tt style="color:rgb(0,102,102);font-size:100%;" >\artauthor</tt> fields are collapsed into a single <tt style="color:rgb(0,102,102);font-size:100%;" >author</tt> field as we deal with the splitting later when it gets parsed in <span tag="MKIV" style="font-style:sans;">Lua</span>. The <tt style="color:rgb(0,102,102);font-size:100%;" >\artauthor</tt> syntax is only kept around for backward compatibility with the previous use of <span tag="MKIV" style="font-style:sans;">bibTEX</span>. <br/>In the new setup we support these variants as well:

<br/>and

<br/>and

<br/>Because internally the entries are <span tag="MKIV" style="font-style:sans;">Lua</span> tables, we also support loading of <span tag="MKIV" style="font-style:sans;">Lua</span> based definitions:

<br/>Files set up like this can be loaded too. The following <span tag="MKIV" style="font-style:sans;">xml</span> input is rather close to this, and is also accepted as input.

<br/>Todo: Add some remarks about loading EndNote and RIS formats, but first we need to complete the tag mapping (on Alan’s plate). <br/>So the user has a rather wide choice of formatting style for bibliography database files.

<br/>You can define unknown commands, or overload existing definitions in the following way:

<br/>Unknown commands do not stall processing, but their names are then typeset in a mono- spaced font so they probably stand out for proofreading. You can access the commands with <tt style="color:rgb(0,102,102);font-size:100%;" >\btxcommand{...}</tt>, as in:

<br/>As this is an undefined command we get: “commands like MySpecialCommand are handled in an indirect way”. <br/>??

<br/>These three suffixes are understood by the loader. Here the dataset has the name <tt style="color:rgb(0,102,102);font-size:100%;" >standard</tt> and the three database files are merged, where later entries having the same tag overload previous ones. Definitions in the document source (coded in <span tag="MKIV" style="font-style:sans;">TEX</span> speak) are also added, and they are saved for successive runs. This means that if you load and define entries, they will be known at a next run beforehand, so that references to them are independent of when loading and definitions take place.

<br/>In this document we use some example databases, so let’s load one of them now:

<br/>You can ask for an overview of entries in a dataset with:

<br/>this gives:

<br/>You can set the current active dataset with

<br/>but most publication-related commands accept optional arguments that denote the dataset and references to entries can be prefixed with a dataset identifier.. More about that later.

<br/><tt style="color:rgb(0,102,102);font-size:100%;" >abstract</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >address</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >annotate</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >assignee</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >author</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >bibnumber</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >booktitle</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >chapter</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >comment</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >country</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >day</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >dayfiled</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >doi</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >edition</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >editor</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >eprint</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >howpublished</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >institution</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >isbn</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >issn</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >journal</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >key</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >keyword</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >keywords</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >language</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >lastchecked</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >month</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >monthfiled</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >names</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >nationality</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >note</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >notes</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >number</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >organization</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >pages</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >publisher</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >revision</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >school</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >series</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >size</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >title</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >type</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >url</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >volume</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >year</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >yearfiled</tt> <br/>If you want to see what publications are in the database, the easiest way is to ask for a complete list:

<br/>This gives:1 Hagen, H. and Otten, T. (1996). Typesetting education documents2 Scarso, L. (2021). Designing high speed trains3 author (year). title pages p. <br/>The rendering itself is somewhat complex to set up because we have not only many different standards but also many fields that can be set up. This means that there are several commands involved. Often there is a prescribed style to render bibliographic descriptions, for example <tt style="color:rgb(0,102,102);font-size:100%;" >apa</tt>. A rendering is setup and defined with:

<br/>And a list of such descriptions is generated with:

<br/>A dataset can have all kind of entries: <br/><tt style="color:rgb(0,102,102);font-size:100%;" >article</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >book</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >booklet</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >conference</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >inbook</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >incollection</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >inproceedings</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >manual</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >mastersthesis</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >misc</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >phdthesis</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >proceedings</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >techreport</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >unpublished</tt> <br/>Each has its own rendering variant. To keep things simple we have their settings separated. However, these settings are shared for all rendering alternatives. In practice this is seldom a problem in a publication as only one rendering alternative will be active. If this be not sufficient, you can always group local settings in a setup and hook that into the specific rendering.

<br/>Examples of list variants are: <br/><tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxlistvariant : artauthor</tt>

<br/><tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxlistvariant : author</tt>

<br/><tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxlistvariant : editor</tt>

<br/>The exact rendering of list entries is determined by the <tt style="color:rgb(0,102,102);font-size:100%;" >alternative</tt> key and defaults to <tt style="color:rgb(0,102,102);font-size:100%;" >apa</tt> which uses definitions from <tt style="color:rgb(0,102,102);font-size:100%;" >publ-imp-apa.mkiv</tt>. If you look at that file you will see that each category has its own setup. You may also notice that additional tests are needed to make sure that empty fields don’t trigger separators and such. <br/>There are a couple of accessors and helpers to get the job done. When you want to fetch a field from the current entry you use <tt style="color:rgb(0,102,102);font-size:100%;" >\btxfield</tt>. In most cases you want to make sure this field has a value, for instance because you don’t want fences or punctuation that belongs to a field.

<br/>There are three test macros:

<br/>An extra conditional is available for testing interactivity:

<br/>In addition there is also a conditional <tt style="color:rgb(0,102,102);font-size:100%;" >\btxinteractive</tt> which is more efficient, although in practice efficiency is not so important here. <br/>There are three commands to flush data:

<br/>Normally you can use <tt style="color:rgb(0,102,102);font-size:100%;" >\btxfield</tt> or <tt style="color:rgb(0,102,102);font-size:100%;" >\btxflush</tt> as derived fields just like analyzed author fields are flushed in a special way. <br/>You can improve readability by using setups, for instance:

<br/>Keep in mind that normally you don’t need to mess with definitions like this because standard rendering styles are provided. These styles use a few helpers that inject symbols but also take care of leading and trailing spaces:

<br/>So, the previous example setup can be rewritten as:

<br/>There is a special command for rendering a (combination) of authors:

<br/>Instead of the last one you can also use:

<br/>You can use a (configurable) default or pass directives: Valid directives are

<br/>The first argument is optional.

<br/>You can tune the way a citation shows up:

<br/>Here we sort the authors and color the citation:

<br/>For reasons of backward compatibility the <tt style="color:rgb(0,102,102);font-size:100%;" >\cite</tt> command is a bit picky about spaces between the two arguments, of which the first is optional. This is a consequence of allowing its use with the key specified between curly brackets as is the traditional practice. (We do encourage users to adopt the more coherent <span tag="MKIV" style="font-style:sans;">ConTEXt</span> syntax by using square brackets for keywords and reserving curly brackets to regroup text to be typeset.) <br/>The <tt style="color:rgb(0,102,102);font-size:100%;" >\citation</tt> command is synonymous but is more flexible with respect to spacing of its arguments:

<br/>There is a whole bunch of cite options and more can be easily defined.

<br/>Because we are dealing with database input and because we generally need to manipulate entries, much of the work is delegated to <span tag="MKIV" style="font-style:sans;">Lua</span>. This makes it easier to maintain and extend the code. Of course <span tag="MKIV" style="font-style:sans;">TEX</span> still does the rendering. The typographic details are controlled by parameters but not all are used in all variants. As with most <span tag="MKIV" style="font-style:sans;">ConTEXt</span> commands, it starts out with a general setup command:

<br/>On top of that we can define instances that inherit either from a given parent or from the topmost setup.

<br/>But, specific variants can have them overloaded: <br/><tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : author</tt>

<br/><tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : authornum</tt>

<br/><tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : authoryear</tt>

<br/><tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : authoryears</tt>

<br/><tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : doi</tt>

<br/><tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : key</tt>

<br/><tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : none</tt>

<br/><tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : num</tt>

<br/><tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : page</tt>

<br/><tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : serial</tt>

<br/><tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : short</tt>

<br/><tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : type</tt>

<br/><tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : url</tt>

<br/><tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : year</tt>

<br/>You can overload such setups if needed, but that only makes sense when you cannot configure the rendering with parameters. The <tt style="color:rgb(0,102,102);font-size:100%;" >\btxcitevariant</tt> command is one of the build in accessors and it calls out to <span tag="MKIV" style="font-style:sans;">Lua</span> where more complex manipulation takes place if needed. If no manipulation is known, the field with the same name (if found) will be flushed. A command like <tt style="color:rgb(0,102,102);font-size:100%;" >\btxcitevariant</tt> assumes that a dataset and specific tag has been set. This is normally done in the wrapper macros, like <tt style="color:rgb(0,102,102);font-size:100%;" >\cite</tt>. For special purposes you can use these commands

<br/>But don’t expect too much support for such low level rendering control. <br/>Unless you use <tt style="color:rgb(0,102,102);font-size:100%;" >criterium=all</tt> only publications that are cited will end up in the lists. You can force a citation into a list using <tt style="color:rgb(0,102,102);font-size:100%;" >\usecitation</tt>, for example:

<br/>This command has two synonyms: <tt style="color:rgb(0,102,102);font-size:100%;" >\nocite</tt> and <tt style="color:rgb(0,102,102);font-size:100%;" >\nocitation</tt> so you can choose whatever fits you best.

<br/>The entries are collected in datasets and each set has a unique name. In this document we have the set named <tt style="color:rgb(0,102,102);font-size:100%;" >example</tt>. A dataset table has several fields, and probably the one of most interest is the <tt style="color:rgb(0,102,102);font-size:100%;" >luadata</tt> field. Each entry in this table describes a publication:

<br/>You can loop over the entries using regular <span tag="MKIV" style="font-style:sans;">Lua</span> code combined with <span tag="MKIV" style="font-style:sans;">MkIV</span> helpers:

<br/>This results in:

<br/>Once a dataset is accessible as <span tag="MKIV" style="font-style:sans;">xml</span> tree, you can use the regular <tt style="color:rgb(0,102,102);font-size:100%;" >\xml...</tt> commands. We start with loading a dataset, in this case from just one file.

<br/>The dataset has to be converted to <span tag="MKIV" style="font-style:sans;">xml</span>:

<br/>The tree is now accessible by its root reference <tt style="color:rgb(0,102,102);font-size:100%;" >btx:tugboat</tt>. If we want simple field access we can use a few setups:

<br/>The two setups are predefined in the core already, but you might want to change them. They are applied in for instance:

<br/>Here is another example:

<br/>A more extensive example is the following. Of course this assumes that you know what <span tag="MKIV" style="font-style:sans;">xml</span> support mechanisms and macros are available.

<br/>The original data is stored in a <span tag="MKIV" style="font-style:sans;">Lua</span> table, hashed by tag. Starting with <span tag="MKIV" style="font-style:sans;">Lua</span> 5.2 each run of <span tag="MKIV" style="font-style:sans;">Lua</span> gets a different ordering of such a hash. In older versions, when you looped over a hash, the order was undefined, but the same as long as you used the same binary. This had the advantage that successive runs, something we often have in document processing gave consistent results. In today’s <span tag="MKIV" style="font-style:sans;">Lua</span> we need to do much more sorting of hashes before we loop, especially when we save multi--pass data. It is for this reason that the <span tag="MKIV" style="font-style:sans;">xml</span> tree is sorted by hash key by default. That way lookups (especially the first of a set) give consistent outcomes.

<br/>Many standards exist and support for other renderings may be added to the core. Interested users are invited to develop and to test alternate standard renderings according to their needs. <br/>Todo: maybe a list of categories and fields.

<br/>The most complex to deal with are the fields that contain names of people. At some point it might be needed to split a combination of names into individual ones that then get split into title, first name, optional inbetweens, surname(s) and additional: <tt style="color:rgb(0,102,102);font-size:100%;" >Prof. Dr. Alfred B. C. von Kwik Kwak Jr. II and P. Q. Olet</tt> is just one example of this. The convention seems to be not to use commas but <tt style="color:rgb(0,102,102);font-size:100%;" >and</tt> to separate names (often each name will be specified as lastname, firstname). <br/>We don’t see it as challenge nor as a duty to support all kinds of messy definitions. Of course we try to be somewhat tolerant, but you will be sure to get better results if you use nicely setup, consistent databases. <br/>Todo: maybe some examples of bad.

<br/>For <span tag="MKIV" style="font-style:sans;">MkIV</span> the modules were partly rewritten and ended up in the core so the two commands were no longer needed. The overhead associated with the automatic loading of the bibliography macros can be neglected these days, so standardized modules such as <tt style="color:rgb(0,102,102);font-size:100%;" >bib</tt> are all being moved to the core and do not need to be explicitly loaded. <br/>The first <tt style="color:rgb(0,102,102);font-size:100%;" >\setupbibtex</tt> command in this example is needed to bootstrap the process: it tells what database has to be processed by <span tag="MKIV" style="font-style:sans;">bibTEX</span> between runs. The second <tt style="color:rgb(0,102,102);font-size:100%;" >\setuppublications</tt> command is optional. Each citation (tagged with <tt style="color:rgb(0,102,102);font-size:100%;" >\cite</tt>) ends up in the list of publications. <br/>In the new approach we no longer use <span tag="MKIV" style="font-style:sans;">bibTEX</span>so we don’t need to setup <span tag="MKIV" style="font-style:sans;">bibTEX</span>. Instead we define dataset(s). We also no longer set up publications with one command, but have split that up in rendering-, list-, and cite-variants. The basic <tt style="color:rgb(0,102,102);font-size:100%;" >\cite</tt> command remains. The above example becomes:

<br/>So, we have a few more commands to set up things. If you intend to use just a single dataset and rendering, the above preamble can be simplified to:

<br/>But keep in mind that compared to the old <span tag="MKIV" style="font-style:sans;">MkII</span> derived method we have moved some of the options to the rendering, list and cite setup variants. <br/>Another difference is now the use of lists. When you define a rendering, you also define a list. However, all entries are collected in a common list tagged <tt style="color:rgb(0,102,102);font-size:100%;" >btx</tt>. Although you will normally configure a rendering you can still set some properties of lists, but in that case you need to prefix the list identifier. In the case of the above example this is <tt style="color:rgb(0,102,102);font-size:100%;" >btx:document</tt>.

<br/>This then permits loading a database (into a dataset) with the command:

<br/>The <tt style="color:rgb(0,102,102);font-size:100%;" >myformat</tt> suffix is recognized automatically. If you want to use another suffix, you can do this: