Searching programatically

While we have indexes for all built-in TMDM structures, we often find that we need to do more fine grained and high performance searches.

You can therefor plug indexes into our indexing engine that will let an index know whenever an item it is interested in is 1) created, 2) updated, and 3) deleted.

>>> import ztm.topicmaps
>>> topicmap = ztm.topicmaps.topicmap.TopicMap()
>>> topic1 = topicmap.createTopic()
>>> topic2 = topicmap.createTopic()
>>> # No topics
>>> list(topicmap.searchResults())
[]

Adding and removing indexes

Indexes are registered as both named and unnamed utilities with the topic map’s closest persistent component registry (site manager). They are registered for the interfaces specified in their registered attribute. The registrations are not used internally. It is so that you can look them up through the component architecture.

You can find all indexes in the indexdrawer which is the ‘indexes’ attribute on all topic map objects.

>>> typeindex = ztm.topicmaps.index.type.TypeIndex(topicmap, configuration={})
>>> topicmap.indexes.addIndex(typeindex, name='types')

The name is used to address the index later.

If you are writing generic code, i.e. code in the ztm.* packages that must work across applications, you should always look up the name of the index you wish to use by looking for an index using the interface:

>>> import zope.component
>>> cr = zope.component.getSiteManager()
>>> utility = cr.getUtility(ztm.topicmaps.index.interfaces.ITypeIndex)
>>> utility.__name__
'types'

Querying indexes

When querying an index you must send in a dictionary or value that matches the expected input of the index. Most indexes have fairly intuitive shortcuts. Please consult the documentation for each index or look for usage examples in code.

Here is an example using the standard type index:

>>> #First some setup
>>> typetopic1 = topicmap.createTopic()
>>> typetopic1.addInstance(topic1)
>>> typetopic1.addInstance(topic2)
>>> typetopic2 = topicmap.createTopic()

>>> # topic1 gets two types
>>> typetopic2.addInstance(topic1)

>>> results = topicmap.searchResults(types=typetopic1)
>>> topic1 in results.topics()
True
>>> topic2 in results.topics()
True

Sorting

Indexes implementing ISorting can be used to sort a resultset. You choose which index to use by specifying the sortindex parameter:

>>> nameindex = ztm.topicmaps.index.name.SortNameIndex(topicmap, {})
>>> topicmap.indexes.addIndex(nameindex, 'name')
>>> results = topicmap.searchResults(types=typetopic1, sortindex='name')
>>> name_type = topicmap.createTopic()
>>> name_type.addSubjectIdentifier(ztm.topicmaps.psis.model.topic_name)
'http://psi.topicmaps.org/iso13250/model/topic-name'
>>> name1 = topic1.createName(u'first', ztm.topicmaps.psis.model.topic_name)
>>> name2 = topic2.createName(u'second', ztm.topicmaps.psis.model.topic_name)
>>> topic1.__name__ = 'first'
>>> topic2.__name__ = 'second'
>>> list(results.topics())
[<Topic first>, <Topic second>]

The search can be reversed with the reversed attribute:

>>> results = topicmap.searchResults(types=typetopic1, sortindex='name', reversed=True)
>>> list(results.topics())
[<Topic second>, <Topic first>]

We can do unicode collation through ztm.icu, but at the time of writing it is not very comprehensive and bound to a particular language.

The default SortNameIndex does collation.

Mandatory indexes

Mandatory indexes are used to implement security filters. These filters are regular indexes that return the set of topics that the principal is allowed to see so that it can be intersected against the result set.

How to implement an index:

Indexes must subclass ztm.topicmaps.index.BaseIndex to appear in the GUI for managing the indexes. The baseclass also implements some common methods.

First decide what you are going to index: Topics, Names, Variants, Occurrences, Roles or Associations.

The hard work is done by apply the apply method. If the index is used by the query, this method is called with the value of that query.

apply must return an II.Set() or II.TreeSet() of topic serials.

Indexing and unindexing

Indexes must have a pair of methods for each type information item.

What these methods do or not do is up to the implementation and purpose of your index. Please take a look at some existing indexes to get an idea of how they work.

An index doesn’t have to use the standard methods. It may subscribe to synchronous events instead and keep itself up to date this way.

Grouping with the index

Indexes implementing the IGroupableIndex can be used to group resultsets. Typical examples are things like types or the first letter of the title.

To use the grouping functionality one must manually call the ‘group’ method of the index.

Searching asynchronously

Warning

The asynchronous API and datastructure is not stable. Should not be used outside of ztm.* packages.

TBD