03 tokens

Introduction

Package tokens could be loaded via the standalone binary, or in Lua with require("aprilann.tokens").

A Token is an abstract C++ class which has different specializations for different tasks and purposes. This class and its specializations are binded to Lua, allowing Lua scripts to be a glue language between C++ algorithms which are developed over the Token abstraction.

In this way, the current implementation of Artificial Neural Networks (ANNs) receives as input a token and produce a as output a token. Normally, for ANNs, this token contains a matrix, but it is also possible to wrap matrix.sparse objects and token vectors. Every ANN component checks the token type, and specialized computations could be done depending in the token type.

From Lua side, matrix instances can be given and taken as tokens.

Abstract class: tokens.base

This class defines the basic interface shared for all token types. It is an abstract class which couldn't be instantiated in Lua. If you try, you will see the following message:

> tokens.base()
tokens.base:tokens.base: Abstract class!!!
stack traceback:
	[C]: in function 'base'
	stdin:1: in main chunk
	[C]: in ?

Usually, C++ method calls return an instance to the class tokens.base, but in some cases it is possible to transform the generic reference to any of the child classes, or in other cases, it is possible to retrieve the contained data.

The transformation between tokens and matrix instances is performed automatically by the glue code between C++ and Lua.

Interface methods

The following methods are in the generic interface of tokens.base class.

clone

token = t:clone()

The method clone is implemented in all the tokens, and returns a deep copy of the caller.

Token types

Token bunch vector

tokens.vector.bunch

The token bunch vector is an array of tokens. Some ANN components allow to receive a vector of tokens which could contain sparse vectors.

> t = tokens.vector.bunch()
> t = tokens.vector.bunch(10)
> t = tokens.vector.bunch{ t1, t2, ... }

clear

t = t:clear()

size

number = t:size()

set

t = t:set(position, token)

> = t:set(1, matrix(2,3):linear())

push_back

t = t:push_back(token)

> = t:push_back(matrix(1,2):linear())

at

token = t:at(position)

> = t:at(1)
> = t:at(2)

iterate

lua iterator = t:iterate()

> for i,v in t:iterate() do print(i) print(v) end