First off, I want to make clear the distinction between a token type and a data type. A token type defines how the token may be used in the editor; a data type defines how the value may be used in a calculation. Of course, the editor will check both types for validity, but they remain distinct. For example, a token type of D might refer to any read-write variable, regardless of its data type.
These are the currently defined token types:
A: the Activation token; this type may not be necessary
C: a constant of some sort; a very useful token type
D: read/write data
E: read-only data
F: function
I: inversion operation
M: macro
N: new equation, again a possibly unnecessary token type
O: operator
S: subsitute
U: unspecified, again a possibly unnecessary token type
The A, N, and U token types each apply to a single token that is already specified by a constant; therefore, I can probably dispense with them. Perhaps I can replace all of these with a G data type, referring to a grammatical token. On the other hand, the D, E, F, S, M, and O token types are surely of value; I will retain them.
A possible new token type would indicate that a token is a procedure, namely, a left-side token with no equation assigned to it. This would differentiate it from a function, so I suppose that I'll need a token type P. I'll have to check whether the inversion token type is used anywhere.
The data types are Anything, Boolean, Event , Location, Number, Person, Thing, Verb. The problem is, I try to indicate both token types and data types with single characters, a gross blunder. This situation clearly calls for an enumerated type instead of a character. The catch is, I use the first character of the type label to declare the data type for all my tokens. This is wrong; I must correct it with more elaborate code.