Running Open Genera 2.0 on Linux

Table of Contents

genera_screenshot.png

1 Introduction

From the early 1980s to the early 1990s, Symbolics, Inc. produced a line of workstations designed to run a highly advanced Lisp environment called "Genera". What made these Lisp Machines so special was the combination of the powerful software running on top of very specialized hardware. The hardware, for example, performed array bounds checking and operated on data as types rather than as flat fields of bits the way other general-purpose computers did (and still do). The Genera environment, in turn, provided workstation-level functionality with a large, high resolution bitmapped graphics display, overlapping windows, and mouse control. In the early 1980s, before even the Macintosh, this was truly advanced stuff.

In the early 1990s, Symbolics took a bold step by releasing the Genera software environment, renamed "Open Genera", on Digital Equipment Corporation's Alpha workstations. To make this happen, they created a "Virtual Lisp Machine" (VLM) that ran under OSF/1 (later, Digital UNIX) and emulated the full Symbolics Lisp Machine architecture.

The source code for VLM was written in C, and as it happens, it was leaked to the public in the early 2000s. Soon thereafter, it was ported to 64-bit Linux.

And here we reach a grey area. Symbolics was dissolved in the mid 1990s, and the current state of its intellectual property is muddled. The rights holders of VLM and Genera itself are not well understood. There is no longer any "Symbolics, Inc."

With that in mind…

This document is meant to guide you through installing the VLM and Symbolics Open Genera on a modern 64-bit Linux system. If you choose to do so, please be aware that you may or may not be violating someone's copyrights. Have fun and play safe!

2 Required Reading

If you're unfamiliar with Genera, the following are required reading.

  1. Genera Concepts (pdf)
  2. Genera User's Guide (pdf)

In addition to these two titles, I highly recommend the book Lisp Lore: A Guide to Programming the Lisp Machine.

3 Install Linux

VLM should run on any modern amd64-bit Linux variant. These instructions specifically target Ubuntu. I've tested on both Ubuntu 15.10 and Ubuntu 16.04.

Ubuntu was only chosen because I have the most familiarity with it. These instructions should also work on Debian 8, with only minor variation (if any).

4 Install Required Packages

You need curl to easily download the required packages, so let's install that first.

$ sudo apt-get install curl

5 Make a Work Directory for the VLM and Genera files

Really, this could be anywhere. I just put it in my home directory for ease of use.

$ mkdir $HOME/genera

6 Install the VLM

The VLM is actally just a single binary named genera. It's the entire virtual Lisp Machine emulator in one file. While you can compile from source code if you can find it, I don't recommend it. Just grab the binary.

$ cd $HOME/genera
$ curl -O https://loomcom.com/genera/genera
$ chmod a+x genera

7 Grab the World and Debugger files

Genera itself is distributed as two components:

  1. A "world" file, which contains the entire running state of a Lisp machine serialized to disk. This file is named Genera-8-5-xlib-patched.vlod (but may be renamed anything)
  2. Files on the local UNIX filesystem exported via NFS. These files live under /var/lib/symbolics

This procedure will grab the required files and put them in the right place.

$ cd $HOME/genera/
$ curl -O https://loomcom.com/genera/worlds/Genera-8-5-xlib-patched.vlod
$ curl -O https://loomcom.com/genera/worlds/VLM_debugger
$ curl -O https://loomcom.com/genera/worlds/dot.VLM
$ mv dot.VLM .VLM
$ cd /var/lib
$ sudo curl -O https://loomcom.com/genera/var_lib_symbolics.tar.gz
$ sudo tar xvf var_lib_symbolics.tar.gz

Finally, we want the symbolics directory to be owned by the user that's going to run genera. In a typical Ubuntu installation, this will be UID 1000 and GID 1000, but please be sure to use the correct values here. Also note that this should match the UID and GID used in setting up NFS below.

$ sudo chown -R <user>:<group> symbolics

8 Set Up the Hosts File

Edit /etc/hosts to match the internal network you're going to put the VLM on. These IP addresses can be changed to anything you like, and in fact some pretty advanced networking is possible, but for simplicity I recommend sticking with these IP addresses. If you use different IP addresses, at the very least you'll need to edit your .VLM file.

You may choose to name genera-vlm and genera anything you like. The names are not that important, but you MUST remember to use the right names later on when setting up your Lisp Machine!

The VLM's Genera software will use 192.168.2.2, and the UNIX host will create a tun0 interface bound to 192.168.2.1.

192.168.2.1    genera-vlm
192.168.2.2    genera

9 Configure time and daytime services

Genera synchronizes its clock with the UNIX host by using the old-school time and daytime UNIX facilities. These are turned off (not even installed!) by default on modern Linux, so you'll need to get them and enable them.

sudo apt-get install inetutils-inetd

Then edit the file /etc/inetd.conf

sudo vi /etc/inetd.conf

Replace with the contents:

time      stream  tcp  nowait root internal
time      dgram   udp  wait   root internal
daytime   stream  tcp  nowait root internal
daytime   dgram   udp  wait   root internal

Then restart the service.

$ sudo systemctl restart inetutils-inetd.service

Verify that it's working by telnet localhost 37 and telnet localhost 13

10 Install and configure NFS

Genera accesses files on the local host via NFS.

First we install NFS with:

$ sudo apt-get install nfs-common nfs-kernel-server

Then configure. You should replace the values forq anonuid and anongid here with the actual user's UID and GID that you are going to use to run the genera binary. On a typical Ubuntu installation, the primary user is UID 1000 and GID 1000, but don't count on that. Verify it.

Edit /etc/exports

/       genera(rw,sync,no_subtree_check,all_squash,anonuid=1000,anongid=1000)

Restart NFS

$ sudo service nfs-kernel-server restart

Then, to export the filesystem, type

$ sudo exportfs -rav

11 Create The tap0 Interface

We're going to let Genera access the host's NFS export through an address assigned to a TUN/TAP interface.

On Linux this is pretty simple.

# sudo ip tuntap add dev tap0 mode tap
# sudo ip addr add 192.168.2.1/24 dev tap0
# sudo ip link set dev tap0 up

Genera will assign itself the address 192.168.2.2, and expect to find NFS files on 192.168.2.1.

12 Start Genera

Now you're ready to start Genera for the first time!

From $HOME/genera, just type:

$ ./genera

And you should be greeted by the DISTRIBUTION world. It'll look like this:

genera_screenshot.png

13 Login and Define Your Site

Important!: If you make a mistake while typing, use your DELETE key, not Backspace, to go back up. This is normal for Genera!!

The world you've just booted is a distribution world. There's already a lot you can do with it, but it's better to define a "Site" and then save a working world. That way you can keep your pristene distribution world as a kind of backup in case you ever want to get started again.

Note that this process writes files to NFS, so it will modify files under genera-vlm:/var/lib/symbolics/sys.sct/. If you really want to start over fresh and clean, you'll need to delete /var/lib/symbolics and unpack it again from the original tar file.

13.1 Login

When Genera starts up, you should see a prompt that looks like:

Please login.
Command:

The first thing you need to do is log in. There's one special global user defined named LISP-MACHINE, so that's the account we'll log into.

At the Command: prompt, type

login lisp-machine

You'll see that Genera helpfully prompts you with (user name) after you type login. Just one of the many ways Genera is looking out for you.

13.2 Define your Site

Next, you'll need to actually define the site. At the Command: prompt, type:

Define Site

Genera will respond by filling in the text (site name), after which you can enter a name for your site. This should be a single word (I've chosen "gecko" below). Then press Return.

Now right-click on

Namespace Server Name: the name of the primary namespace server

Enter genera here and press Return. Genera will respond by adding several more editable fields.

Right-click on

Unix Host Name: the name of the DEC-AXP host on which Open Genera is running

This time, enter genera-vlm.

After doing both of these, click on END (or press the End key) to save your changes. You now have a site defined!

The dialog should look something like this:

define-site.png

Now that you've defined a site, it's time to save the world.

14 Saving the World

14.1 One-Time Setup

After defining a site, the ip address for host "genera" (or whatever you called your 192.168.2.2 LISPM) will be wrong. It gets set to the external IP address of your Linux host. It must be set to 192.168.2.1 or you won't be able to re-load your saved world without a lot of inconvenient futzing.

To edit the host, type:

Edit Namespace Object Host genera

And change the Address to

INTERNET 192.168.2.2 tap0

Now type

Save Object

And answer 'Yes' to the prompt. Finally, type Quit to get back to the top level Lisp listener.

This must be done before you save the world, otherwise it will be unable to reach the local network correctly when you restore from it!

14.2 Saving the World

Now you're ready to save your working world.

At the Command: prompt, type:

Save World

When you hit 'space' after this, you'll be prompted with the text

Save World (on file [default GENERA-VLM:Genera-8-5.vlod])

That's the default world name, but you can use anything. For example, I named my world GENERA-VLM:Genera-8-5-working.vlod. Type that in, then press Enter.

You'll be prompted for confirmation. Type Y and Enter.

At this point, Genera will reboot itself. But it's not ready to use! You need to halt Genera, edit your .VLM file, and restart it.

14.3 Halt Genera

Type

Halt Genera

at the prompt. You'll be asked for a couple of confirmations. At this point the debugger window should pop up and ask you for a third confirmation that you really want to exit Lisp (if it doesn't, find the hidden window and type 'Y' at it. This behavior depends on your enviornment and window manager.)

You should now be back in Linux. It's time to edit your .VLM file.

14.4 Edit .VLM file

Open your .VLM file, and comment out the line genera.world: Genera-8-5-xlib-patched.vlod. Now add a new line that reads:

genera.world: Genera-8-5-working.vlod

(Or whatever you named your working world VLOD file)

At this point you can restart genera by typing ./genera.

Congratulations! You're in a working Genera world. It's time to explore.

15 Useful Genera Knowledge

15.1 Shutting Down

To shut down, type

Halt Genera

You'll be prompted a couple times for confirmation. After the Genera window closes, you'll have to confirm again in the Cold Load window.

15.2 Key Mapping

Symbolics Lisp Machines came with really amazing keyboards just covered with modifier keys.

When Symbolics released the VLM, they realized they had a problem. DEC Alpha workstations used PS/2 style keyboards, so they had to somehow map all of their modifer keys to PS/2 keyboard codes, and build a converter box to use their keyboards with the DEC hardware.

Thus, this mapping was born. You can get away with using a regular 101-key or 104-key PC keyboard, as long as you follow this mapping. (N.B.: "KP" means the keys on the number pad)

key_mapping.png

15.3 Scrolling

I always forget how to scroll in a Genera window if I haven't been using it for a while. It's pretty simple.

Key Description
Left Moves the line indicated by the pointer to the top of the screen.
sh-Left Moves the line indicated by the pointer to the bottom of the screen.
Middle Displays the percentage of the pane contents that approximately corresponds to the position indicated by the pointer.
Right Moves the line currently at the top of the screen to the position indicated by the pointer.

15.4 Setting up Who-Calls

When a site is defined, Genera calls (si:enable-who-calls :new) automatically for you.. But we want to enable who-calls on all functions, not just new functions, so we must manually set that up

(si:enable-who-calls :all)

This takes a few minutes to run.

15.5 Finding functions

To search for all functions that have the string "crypt" in their name, one can simply type

(apropos "crypt")

15.6 Who Calls

After "Who Calls" has been set up as above, you can find all callers of 'rpc::encrypted-password)

(who-calls 'rpc::encrypted-password)

15.7 Authentication

UNIX authentication seems to be defined in the flavor RPC::UNIX-AUTHENTICATION-MIXIN. Specifically, see the definition of the function RPC::USERNAME-AND-PASSWORD-VALID-P

Here's the original definition.

(defun-in-flavor (username-and-password-valid-p unix-authentication-mixin)
         (username password)
  (declare (values username-valid-p password-valid-p))
  (if (null username)
      (values nil nil)
    (multiple-value-bind (encrypted-password defaulted-p)
    (let ((*unix-authentication-allow-defaulting* t))
      (username->password (unix-name-lookup-access-path) username))
      (cond ((and defaulted-p
                  (not (string-equal username "anonymous"))
		          (not (string-equal username "lisp-machine"))
		          (not (string-equal username "nobody")))
	         (values nil nil))
	        ((or (zerop (string-length encrypted-password))
                 (and password
		              (string= (unix-crypt password encrypted-password)
			                   encrypted-password)))
	         (values t t))
	        (t
	         (values t nil))))))

We can hack this function to overcome the need for crypt-style passwords in NIS authentication. We could do a couple of things here:

  1. Implement MD5, SHA256, or SHA512 encryption
  2. Hack around (unix-crypt) by editing this function definition and adding a no-op of some kind
  3. Figure out Genera's Flavor system and actually implement our own login mixin (probably the right way?)

What I personally have done is to modify the function so that I can log in with the password "xyzzy". This is very silly, but it works fine. It's also a good introduction to hacking Genera.

Here's how I changed mine. Notice the (string= "xyzzy" password).

(defun-in-flavor (username-and-password-valid-p unix-authentication-mixin)
         (username password)
  (declare (values username-valid-p password-valid-p))
  (if (null username)
      (values nil nil)
    (multiple-value-bind (encrypted-password defaulted-p)
    (let ((*unix-authentication-allow-defaulting* t))
      (username->password (unix-name-lookup-access-path) username))
      (cond ((and defaulted-p
                  (not (string-equal username "anonymous"))
                  (not (string-equal username "lisp-machine"))
                  (not (string-equal username "nobody")))
             (values nil nil))
            ((or (zerop (string-length encrypted-password))
                 (string= "xyzzy" password)
                 (and password
                      (string= (unix-crypt password encrypted-password)
                               encrypted-password)))
             (values t t))
            (t
             (values t nil))))))

16 Appendix A: Patching Genera's XLIB

If you downloaded the file Genera-8-5-xlib-patched.vlod you can safely ignore this step. This procedure will modify an un-patched Genera-8-5.vlod file and make it usable.

Open Genera 2.0 has a pretty nasty bug in its X11 implementation that will prevent it from running on modern X implementations, UNLESS you follow this procedure to patch it.

The purpose of this procedure is to patch the Genera-8-5.vlod world file so that it can be run without crashing on Ubuntu 15.10 and higher.

NOTE: The time and daytime services MUST NOT BE RUNNING. Shut them down as a first step. Otherwise, genera will get the time from the system and start running. We don't want that!

To get started, type

$ ./genera

(The VLM needs to create the appropriate network tun0 interface, so it MUST be run as root!)

This will bring up the cold load debugger window. Note, however, that it will be minimized, so you'll have to open it. After about ten seconds, it will prompt you for a date and time. At the prompt, enter the date in the form MM/DD/YYYY HH:MM, for example

2/1/2016 17:58

This will fail, but we WANT it to fail. You should see the prompt:

Error: Unable to set calendar clock.

TIME:SET-CALENDAR-CLOCK
   Arg 0 (TIME:NEW-TIME): 3663356280
s-A, <Abort>: Exit process Initialize Genera
→

At the "→" prompt, you can enter just about any Lisp expression. We're going to use this fact to replace the body of a function, xlib:set-modifier-mapping, with a no-op. This is to prevent X11 from just barfing and failing to render Genera at all.

So, at the → prompt, type:

(defun xlib:set-modifier-mapping (display &rest args) (values))

Note: When you type the first open parenthesis, the cold load debugger will respond with:

Eval (program):

This is normal!

When you type the final closing parenthesis, you will be prompted with:

Warning: Function XLIB:SET-MODIFIER-MAPPING, being redefined,
was previously defined by file SYS:X11;CLX;TEXT. OK? (Y, P, or N)

Just type Y here.

Now, type :Return, with the colon, to exit and continue on. You'll be prompted with:

Return from TIME:SET-CALENDAR-CLOCK? (Y or N)

Just type Y here.

At this point, Genera should load. You'll need to save the modified world right away. To do so, at the Command: prompt, type:

Save World Genera-8-5-xlib-patched.vlod

The world will be saved in the new file Genera-8-5-xlib-patched.vlod

Now type

Halt Genera :shutdown

and answer Y to a few prompts.

Finally, edit the $HOME/genera/.VLM configuration file to point at the new world:

...
#genera.world: Genera-8-5.vlod
genera.world: Genera-8-5-xlib-patched.vlod
...

Author: Seth J. Morabito

Created: 2017-10-08 Sun 18:05

Emacs 25.3.1 (Org mode 8.2.10)

Validate