Indexes v. full text searches

Comment from an email discussion list member

In my opinion, indexing is pointless. Searching is much more important than indexing.

I think you should provide the simplest automatic indexing, based on topic headings, and rely on your users to use full-text search to actually find anything.

Full-text search is easy and it works perfectly. Indexing, on the other hand, depends on the skill and commitment of the indexer. For myself, I feel that my life is too short to spend much of it on indexing.

My response

Yes, full text search (FTS) has its place (and Google et al have a lot to answer for in this regard).

BUT FTS brings with it a lot of garbage as it find any matching words without any reference to their context. You can narrow the results list by judicious use of Boolean operands and syntax such as quote marks, parentheses and the like—but most users don’t know about them or how to use them. However, this still doesn’t put the words into context. In many ways FTS is like a concordance… ‘let’s list every word we find in the document no matter how important it is’. Concordances treat every word equally.

Indexes on the other hand—at least, well-constructed indexes—serve a different purpose in focusing the user’s attention on the words most likely to have relevance.

Well-constructed indexes are done by humans who can detect nuances and meaning of words—in the context of their usage. Indexes created by software (of any kind) just don’t have this sophistication at the moment. A good indexer can create “see” references from unused terms to used terms, and “see also” references to similar terms. Complex indexes also refer users to broader and narrower terms.

Others have mentioned synonyms and they are one of the prime reasons why human indexers are much better at this than software. A FTS can’t distinguish between words such as “editing”, “amending”, “changing”, altering”, “modifying” (and their variations)—yet they could all be used in the document. A human indexer can set one term as the preferred term and refer all other uses to that term, thus offering the user the FULL range of topics that cover anything to do with changing something.

Others have also mentioned that print output (NOT screen versions of print, like PDF) doesn’t have FTS, and a good TOC and Index is the way users access the information. I believe there is a place for both… and I would be very sad to see the day when indexes are relegated to the trash.

Has this tip helped you? saved you time? saved your skin? You can thank me by clicking on the cup and buying me a coffee. (An E-Junkie shopping cart page will open where you can pay for my coffee via PayPal.)

See/See also: And the difference is?

If you’re unsure of the differences between a see and a see also cross-reference in an index, here’s an explanation that may help:

• See: Refers you from an unused term to a used term. For example, ‘ornithology, see birds’ means that anything about birds in this document is listed under birds in the index, not under the more scientific term of ornithology. In the index, only birds has a page number reference.
• See also: Refers you to other used terms related to this one. These terms may be ’siblings’ or ‘children’ of the parent term. For example, you may see ‘birds, see also reptiles’. Both terms are at an equal level, so are ’siblings. However, ‘birds, see also eagles, flamingos, parakeets’ refer to terms that are narrower than the parent term birds, so are ‘child’ terms. All terms are listed in the index with their respective page number references.

[This article was first published in the September 2005 CyberText Newsletter.]

Has this tip helped you? saved you time? saved your skin? You can thank me by clicking on the cup and buying me a coffee. (An E-Junkie shopping cart page will open where you can pay for my coffee via PayPal.)