Index

As this writing is a collection of my ideas that have not been written down in full before now, this page will be gradually updated and refined whenever I recall more and more of my ideas concerning the topic and feel the want. Currently, only some of those I've focused on the most with my recent developments are present.

In brief, I intend to explain the entirety of the history of, ideas behind, and the general design of the Meta-Machine Code tool I've devised and will update this article when I notice omissions and whatnot.

Expanding on mentions of a tool from previous experimentations and discussions of design from thoughts on machine architecture, I debut the tool mused over years ago and mulled over until now. Starting with the history, it was years ago, in a car in a parking lot, in which, where, and when I envisioned a vague machine code, now named 33554432, in which instructions were n-bits encoding n instructions as toggles that enabled or disabled them; the first instruction of sorts envisioned simply prevented an instruction from being executed, which, optimized in hardware, would allow easily self-modifying loops and whatnot; an interesting consequence that was noticed was how, in fitting this to a textual assembler's model, it would be horribly ugly and impractical to use. I then envisioned a tool in which instructions were bound to keys or buttons and arguments and whatnot were handled in an automatically aided fashion. I believe it was even earlier than all of this, but may not have been, that I'd given a decent amount of thought to programming without what is considered syntax through binding keys and other interfaces to computational units and whatnot; the primary issue was that it ``would look like Forth'', in that a bedrock of basic functions could be provided, but the majority of programming would merely reference names of sorts, defeating the purpose, not to mention the unwieldiness of binding hundreds or thousands of keys and whatnot; it was recognized, however, that a machine code, through the generally and relatively low instruction count and label use, could be used effectively in such a way.

With the basic concept in place, I let it sit. I was reading an assembler code listing containing local labels and remarked to myself how much I disliked the convention. I soon began to mull over further reasons I disliked the current offering of assemblers. While I dislike the notion of local labels, I didn't want to justify restricting functionality based on my own preference. I also tend to dislike string syntax and other unchangeable aspects of these tools. I decided that the most reasonable course of action was to take this tool and simplify it to the smallest reasonable base of functionality and then put in place a system that would allow for the implementing of features that were desired on an individual basis. The solution was the concept of a meta-machine code, a machine code likely implemented in a virtual machine by the tool, designed and used for manipulating the ``target'' machine code and so write instruction sequences and whatnot; with such a system in place, local labels, dumb macros, strings, and more powerful features not in any assembler could be built; a particularly important instruction envisioned, not unlike the nothing bit of 33554432, was a disassembling and later also an assembling instruction which would ease inspection and modification of the target; these would influence later integral routines. The original idea was to provide specific meta-machine codes and a general meta-machine code which would be 33554432; with the deemed failure of 33554432 for this case, among others, that left only the specific meta-machine codes, which were suitable enough and influenced by 33554432 in many ways, such as being memory-to-memory systems.

Yet another aspect of assemblers and modern machine code programming in general that I horribly disliked is the splitting of a program into sections and a complete disregard for what the assembled machine code is. Both of these qualities impact self-modifying programs greatly, with segments that can't be both executed and written to, among other restrictions, and with the muddying of what the numerical values of the instructions actually are, discouraging the use of the program as data. In addition to it being simpler, the tool only builds dumps of memory for these reasons; it was figured that a generic binary format header could probably be created for appending if desired and later realized the tool itself could easily create and append this, if desired. So, the tool was also envisioned reasonably early on that it would show several representations for instruction and data, to aid the use of anything as data or instruction.

Comments were a particularly strange feature to consider, as there is no facility in the meta-machine code concept to pass arguments to routines. A meta-machine code program is aware of the target address it was called for, but this can't contain arguments that could be preserved easily, as the result would presumably be sent to that address and having such arguments in a finished program's memory would be poor. Several solutions were devised. For strings and other varying data, two meta-machine code programs could be used; a string program would be called with the address and read information until noticing an ending meta-machine code program attached to an address, signifying that it should cease there; the ending meta-machine code program could do nothing, it's main purpose to signal to its accompanying program, or could be a reverse of the accompanying routine; yet alternatively, the same routine could be used for both. For programs that wouldn't contain arguments in the target memory, yet still need them, a stub program could be attached to an address, instead. This stub program would call a more fundamental meta-machine code program and use a calling convention from which it could get arguments; the main drawback of this approach is the filling of the meta-address space with data that may only be used once, but this is arguably not an issue, especially with larger meta-address spaces. Coming back to comments, an issue persisted; comments contain data that should lie in the meta-address space, yet also don't add anything to the target address space; as the display model used associates all display with an address, this poses the question of how to actually display comments; the expected solution was for a meta-machine code program to advance the display and show the comment there, perhaps; this would limit the size of comments, but wasn't considered to be particularly important. I've grown fond of separating comment and program into separate stores for a time now, compared to the intermingling of them. The ultimate solution, in considering how complex such a scheme would grow, is to simply not have any facility for comments, as they merely complicate matters unnecessarily; it would be much more useful for, say, a separate file to be used for such, associating a line with each address, perhaps.

Approaching displaying, the originally vague nature of the interface began to specify. Considering the nature of producing memory dumps, the tool is considered to be particularly suited to producing ROMs, such as for a video game system. The ability to display sprites as they would be displayed by hardware was considered an important aspect of this and another way in which the tool would be superior to an assembler. The earliest idea of display I curently recall was naught but rows depicting the information stored in the corresponding addresses; ideas of fonts and colors and bitmaps for sprites were present, along with many others. The display was intended to be very filled with information. The size of a row has been rather solidified at two hundred and fifty six rows and two hundred and fifty six columns, at most, with the basic information consuming, say, the first sixty four columns. It was considered that meta-machine code programs could modify the entire row, but this idea was later discarded due to the complexity it added to all basic operations, leaving display solely to the latter rows. The question of where I/O would take place was an important one; disliking the mode-line and minibuffer of Emacs, it was decided that the entire interface would be such described rows and that any I/O would occur in a transient area and disappear when no longer necessary. A ``popup box'' and a comandeering of a row were the two main choices; such a box would appear to collect information and then disappear to reveal the underlying rows; the commandeering of a row would temporarily hijack it for questioning and whatnot and later redisplay the affected row.

The basic functionality of the tool will now be described in brief. The basic functionality includes: displaying the rows of information; accepting input delimited by numerical size or character length; showing output; providing the target address space; handling the entering and resolving of names and the special case of names associated with an address, labels; loading information from external sources; handling the input devices; and providing interfaces for the meta-machine code to access various aspects of this basic functionality.

Follows is a mockup of what the tool was previously thought to maybe have looked with a purely textual interface; this version depicts CHIP-8:

0766-0767 2FE-2FF 0011101000010001 3A11 14865      routine-label Skip next if VA = 017
0768-0769 300-301 0000000011100000 00E0 00224 Clear the screen
0770-0771 302-303 1000101011000001 8AC1 35521 VA <- VA OR VC
0772-0773 304-305 1000000100100100 8124 33060 V1 <- V1 + V2; VF <- overflow
0774-0775 306-307 0000000011101110 00EE 00238 Return
0776-0781 308-30D ................ .... ..... write-me string
0782-0783 30E-30F ???????????????? ???? ????? random ?
0784 310 00011000 18 24 sprite ███ ███
0785 311 00100100 24 36 ██ ██ ██
0786 312 00100100 24 36 ██ ██ ██
0787 313 00011000 18 24 ███ ███
0788 314 00100100 24 36 ██ ██ ██
0789 315 01000010 42 66 █ ████ █
0790 316 11111110 FE 254 local-data This is a comment that uses this extra space!

It was once considered unwieldy to require a meta-machine code routine to exist for each display type, such as one for each instruction or so. This lead to a tangent I deemed ``Pattern-Set Programming''. The idea was to distill what meta-machine code would be doing into something particularly easy to use and much simpler. Pattern-set programming went through many revisions; pattern set programming is also much less tied to the target instruction set. Disassembling the name, pattern-set programming is programming with an ordered and finite set of patterns. Each pattern program has six parts: the type; the mask; the form; the display pattern; a list of questions; and the following pattern program or a termination. The type denotes the instruction's form, with zero reserved for meta-machine code instructions. The mask is the data that will be combined with the final form to produce the final data placed in the address space; to restrict alterations to the mask, any data entered would be entered in a work area, ANDed with the negation of the mask, and then combined with the form by AND. The form is the initial contents of the instruction, typically designating the instruction code with zeroes for the other slots. The display pattern is how the data in the address space will be shown in the interface. A pattern question has four parts: the display question; the size of the data; the position of the data; and the following question or a termination. The display question is what is asked of the user. The size denotes the numerical limits of the answer. The position denotes the contiguous bits that the answer will be placed in. Both questions and pattern-set programs will continue until terminating, with pattern-set programs providing following, contiguous data.

This tangent was terminated once it was realized it was overcomplicating the tool. It was decided that any and all such abstractions over the meta-machine code should both be implemented in the meta-machine code and also generate meta-machine code, acting as compilers of a sort. With special notice towards displaying graphical information such as sprites, it was decided this was too unwieldy to implement in anything but the tool itself, as a special display mode accessible to the meta-machine code as opposed to character display; based on the peculiarities of this mode, a meta-machine code program using it would either be simple or require some basic machinery, it's intended.

An address space of near four kilobytes was originally thought to be too small for a reasonable selection of meta-machine code programs, but this was figured to be wrong. I find it particularly interesting to optimize programs for size, as this is an exact and trustworthy measurement, as opposed to speed. It was noticed that the majority of meta-machine code routines would be performing nearly identical actions and so could be taken and turned into small routines, with most meta-machine code routines then simply loading an argument or so before jumping to the appropriate routine. This was also figured to make program generation far simpler for a system such as Pattern-Set Programming, as it could then mostly emit unintelligent data, rather than need some more complex optimization scheme.

The initial target of the initial Meta-Machine Code tool was to be CHIP-8; this was chosen for its simplicity and small address space, which delayed the need to address certain pressing issues, such as the inability to store the entirety of larger address spaces and so use sparse representations.

Follows is a pattern-set program for CHIP-8; pattern-set programs have no syntax (being entered piecewise by prompt) and so each part is delimited; this encodes a single instruction:

  type     mask form     pattern   question size position question size position
10 F000 6000 "VX <- NNN" "Register" 4 8 "Value" 8 0
binary hexadecimal decimal

The original mulling over a storage format for pattern-set programs can be reused for a meta-machine code implementation rather well. I consider it very important to design numerical formats for data that can't contain invalid data, sans length considerations, and a machine code program is one reasonable way. Originally, questions simply had a mask that could be used to specify answer depositing, but this allowed for non-contiguous bits to be specified and so was discarded for the current scheme, which allows no such thing. Interestingly, the current arrangement consumes less space than a mask would, as two four bit fields can be used, rather than a single sixteen bit field. The addition of variables was considered, to allow for sequences of instructions that share data, but this was easily implemented with simple meta-machine code and it was considered to overcomplicate pattern-set programming to add this. Questions and even sequences of instructions were considered to be too complicated at a point, but lacking this both made it much less useful and limited and so this thinking was reversed.

The following explains the special sequences of characters in a CHIP-8 pattern-set program's display pattern; it was originally intended that these sequences be restricted by instruction type to those that seemed sensical for them, but this was later changed as well, simplifying things further; each sequence was chosen to consume the same amount of space as its largest possible replacement, originally to aid calculating length for output limits:

  CC The first four bit instruction code in decimal.
VX The capital letter V followed by the second four bits denoting a register in hexadecimal.
VY The capital letter V followed by the third four bits denoting a register in hexadecimal.
NNNN The last twelve bits of an instruction in decimal.
NNN The last eight bits of an instruction in decimal.
NN The last four bits of an instruction in decimal.

The instruction responsible for changing the consideration of sequences only being valid for certain instructions is the following:

  type     mask form             pattern       question size position
01 FFF0 00C0 "Scroll down by NN" "Shift amount" 4 0
binary hexadecimal decimal

This simple Super CHIP-8 instruction is the only one using the 0 instruction with the final nibble being an argument of sorts. The alternatives are binding a similar instruction sixteen times, using meta-machine code, or using ``Scroll down by last nibble'' as indirection, none of which seemed better than allowing more flexibility in display.

That brings to another consideration of the tool. It is designed to separate meta-information from information; that is, the machine produces a memory dump of the actual program and, separately, produces the meta-memory dump. For convenience, this could be kept together or, for saving space, could be separated into constituent parts, as names are the most likely to change between programs, whereas the other categories could generally be useful for many programs. The meta-information considered important includes the keyboard tables, the meta-machine code routines, the data associated with an address, and the names; the last two are considered most likely to be stored seperately from the remainder.

The data associated with an address includes a subsuming bit, a name code, and a meta-machine code routine code. This consumes a variable amount of information for each target, but is thought to be reasonably reducible to three or four bytes per each target byte for many machines. Originally, each address would have an integer that determined whether the data in the following bytes would be considered or not, but this complicated display code so much so as to make it roughly equivalent to traversal of a singly-linked list backwards and so bothersome; the single bit serves as a replacement, with one state referring to the beginning of a new range and another signalling that the previous range consumes it. The thinking is that it would be useful to be able to toggle if the first address supresses the display of following ranges, such as hiding data complexity unless explicitly and probably temporarily wanted.

The integral routines or instructions available to a meta-machine code will now be discussed. An assembling and a disassembling routine are particularly important and would be made to consume the output of the other; the assembling routine would accept an instruction type and use this to determine the format of the following locations in memory, to be assembled into an instruction at a given location; the disassembling routine would accept a bare instruction and provide its instruction type, with the following memory locations containing the instruction segments in the predetermined layout. Due to realities of the tool noticed, a verifying routine is also under consideration, which would accept an instruction type and instruction and ensure that this is then a valid instruction, but the specifics of this routine are too vague and the consequences not entirely considered. The twin input routines are then important, one allowing for numeric input and the other for character input, both delimited in their respective units; the numeric routine would include a facility for names to be used in place of numbers and also allow for names to be the default action rather than numbers, if desired. A simple outputting routine allowing for concatenation of output until released is also deemed necessary, along with other special display modes based on the target. Routines for accessing meta-information may or may not be explicitly necessary, based on future layout considerations. Asides from this, programmatic control of the name system is considered important. Lastly, an instruction to return control to the tool is mandatory and simple. This is the extent of special meta-machine considerations with respect to the tool, asides from the memory mapping.

The memory mapping should contain a random number generator, further hooks into the name system, the keyboard tables, and other functionality as seen to be necessary or useful.

The design of names will now be discussed. It is intended that a name should be able to store a number that can hold an entire address or an entire instruction, to be determined when the target is being considered and leaning towards the smaller but biased towards the address, typically. Asides from this, the name has a length, its name itself, and a bit determining if its value is a label and so to be shown when the address is displayed. The design of names posed a particularly interesting problem at a point; there is no clear and necessary relation between a value entered and its representation in the instruction in a form and granularity suitable for the name system's memory unit-based alignment. A great deal of thought was given to the solution; in the case of, say, CHIP-8, the granularity could be reduced to a nibble and the issue largely eliminated, but other architectures, such as RISC-V, complicate this greatly. The current solution is to have each instruction handle names especially and, due to size considerations, to only initially provide this for instructions that manage memory and control flowing, as these are the instructions that will primarily benefit from this. The usage data is intended to allow for updating values without consulting the meta-machine code programs, when changed; based on the architecture, this may require specialized routines to be identified and included with the usage data or, again in the case of CHIP-8, require no particularly intelligent or clever behavior on the behalf of the name system. Touching lightly, the memory maps into the name system explained in brief beforehand are intended to provide access to a meta-machine code routine that would resolve names manually; this would enable a routine adding syntactic rules to names and so, say, providing name arithmetic and local labels and whatnot; complicating this, local labels in particular, was uniqueness of names, and so this was abandoned, favoring name codes that uniquely identify names instead. To finish, a great deal of mulling was also given to how to have the system ask simple yes or no questions; the best solution thought of was far more general and involved asking for numeric input with names as the default and simply having names corresponding to what is desired and so ``yes'' could be one and ``no'' zero, as an example; this then greatly simplifies a great many manner of colorful and pleasant input methods.

To close for now, a great deal of thought has been dedicated to weaker platforms that are still capable of hosting a meta-machine code system. The thinking was that the target machine code could serve as the meta-machine code as well. A virtual machine not written can't be slow. This is the approach being pursued in a more practical version being targetted at a modern machine. This resolves some peculiarities of the design, as a meta-machine code routine can assemble, but not disassemble itself, necessarily, as the meta-machine code may be ambiguously overlapping with the machine code, as with Meta-CHIP-8. This generally has the instructions becoming routines, but this has varying effects on code density and other considerations and is being actively pursued and mulled over. Some interesting results arise, such as the finishing routine that returns control to the tool being a simple jump to the prime loop.