More Grounding Required

I've been trying to invent a general-purpose scripting language lately, and having great difficulty figuring out what exactly it is I'm trying to accomplish and how to do it, let alone actually building anything.

Part of the trouble stems from that there are several different use cases driving my desire for this language (some of which are recursive), and paradigims I want to support in a unified way.

I want it to be self-hosting. I want it to run on the JVM. I also want it to be capable of transpiling to other languages/platforms. I want it to be a Scheme, or at least a Lisp. I want it to be purely functional. I want a TCL-like language for running commands. I want the bootstrap interpreter to be tiny, which points towards a Forth-like language. I want Racket-compatible "#lang" directives so that I can change my mind about syntax without having to build a whole new system. I want to be able to reference functions written in this language as active URI functions, or easily deploy them as web services.

That's a lot of different requirements, some of which are apparently at odds with each other. Forth and Scheme imply specific, incompatible runtime semantics.

Maybe I should spend less time trying to build this perfect unified thing and more time building little prototypes. A big obstacle to that is that package management in the Java ecosystem is a bit of a chore. There's Maven, but the process of actually publishing my packages somewhere has way too many steps.

After months of failing to bootstrap my own Java-based build/package-management tool I decided to bite the bullet and "just use Maven," punting on the package hosting business for now and just mvn install-ing things. I figure I can solve that problem later.

Actions

I've been enfatuated with the idea of separating construction of actions-to-taken from the actual execution of them. This is essentially how things are done in Haskell; an IO x represents some action to be taken, producing a result of type x. Note that the action itself doesn't take any parameters. putStrLn "Hi there" is a call to a function (putStrLn) that creates an IO (). In Haskell-ese, putStrLn :: String -> IO ().

That's enough pseudo-Haskell. I'm doing this in Java. For reasons.

In practice, an implementation of Action that 'knows how to execute itself' might look like this:

interface Action<C,R> {
	public R execute(C context);
}

Alternatively, you could represent actions in some more declarative way, where the runtime knows how to execute actions, rather than the actions knowing how to execute themselves. Or one could have a system that doesn't execute actions at all, but only compiles them, or analyzes them for some other purpose. For this reason it is tempting not to define an interface like the above.

There are a couple of reasons why I think that such an interface definition is useful.

For one, it provides a concrete starting point for an interpreter, without which I tend to get lost in architectural nowhere space and never actually build an implementation of my language.

The other is more interesting to me at the moment. There are a couple of conceptual problems that were bugging me.

Wanting actions to be a pure representation of something-to-be-done, they shouldn't themselves require a parameter in order to execute, I thought. But then how do you represent the action of 'actually do an action'? That itself needs to take a parameter—the action to be taken—and once you've constructed that action, you're left in the same place you started.

A related problem: What if an action is to write to an open file desciptor? Where does that file descriptor come from? A file desciptor is not a 'pure value', so a function can't return it, which means it can't be passed to the action's constructor, which means it can't be baked into the action object itself; a way to work with external objects like file desciptors therefore has to be provided by the context in which the action is run!

Having actions know how to execute themselves and taking a context object as a parameter to the 'execute' method seems to solve these problems, or at least clarify things in my mind. Having that execute method on the Java interface reminds me what it is that these objects are supposed to be able to do, even if I never actually call them. I can always abstract things more after getting a working system fleshed out.

On to the next conceptual problem.

"Why", I would ask myself, "should actions get this special separate-construction-from-execution treatment, when regular functions don't?".

Note that Action<C,R> looks a lot like Function<I,O>. In this way, actions look like functions. I keep the interfaces separate because I want to distinguish between evaluating a pure function and executing an action; the two operations are, while similar, not compatible. But this symmetry could be seen as a solution to the problem (the problem being the apparent asymmetry).

But there is another way of looking at things. Instead of Action being a side-effectful Function that takes the context as its parameter, Functions can be seen as Actions that don't need to care about context due to their purity. Which leads me to conceptualize an alternate version of Function thusly:

interface PureAction<C,I,O> {
	public O execute(C context, I input);
}

Almost an action, but requires an extra parameter. Solution: process the input as a separate step:

/** A function that returns a PureAction; i.e. a PureAction constructor */
interface ActioneyFunction<C,I,O> extends Function<I,PureAction<C,O>> {
	public PureAction<C,O> apply(I input);
}
/** An action that is guaranteed not to do any side-effects */
interface PureAction<C,O> extends Action<C,O> {
	/** No side-effects allowed! */
	public O execute(C context);
}

PureAction is a thunk! Let's write that code again with some new names.

/** A function that returns a Thunk */
interface ThunkFunction<C,I,O> extends Function<I,Thunk<C,O>> {
	public Thunk<C,O> apply(I input);
}
/** An action that is guaranteed not to do any side-effects */
interface Thunk<C,O> extends Action<C,O> {
	/** No side-effects allowed! */
	public O execute(C context);
}

Again, these definitions are to aid my won conceptual understanding; it doesn't necessarily make sense to subclass things in an actual Java codebase just to say "same as parent but no side effects". Or maybe it does, in order that the type checker can help verify things. But I haven't tried it, yet.

So this solves the asymmetry in a slightly different way. In the end, everything is conceptually a function, either a side-effectful one (an Action), or a pure one (a Function). Any time you do anything, you are essentially applying a function, either to an explicit argument, or implicitly to the context.

Having functions return Thunks instead of...bare POJOs representing values, I guess, is that it provides another point of potential indirection. Maybe you have an expression like concat( expensive operation, another expensive operation ) and want to lazily evaluate it, representing the concept of the concatenation of those two sub-expressions without actually doing the work to evaluate them, first. Thunk gives you a place to put metadata about the value such as "not actually calculated, yet; if you want the actual bytes, call method X". In this way it is similar to Action; Action gives you a way to talk about an action to be executed without having to actually execute it first, and Thunk gives you a way to talk about values without having to actually have a concrete representation of the value on hand.

Such a Thunk's execute method might actually calculate the value, but use the passed-in context to cache values or request external (but otherwise pure/immutable) resources. Maybe the context in that case should be a different type; ReadonlyContext or somesuch, which would lack methods to mutate anything in a way that's externally visible.

The thunk/action abstractions would normally live at different levels, explicit Actions being a step above Thunk. But a language could choose not to enforce that layering. A language that allows functions to have side effects could be implemented in the same terms as one that does not just by dropping the reqirement that thunks are side-effect-free. In such a language, putStrLn "Hello" would return a thunk that, when executed, actually does the action, whereas in a Haskell-like language, that thunk would return an action that, when executed, would actually do the work. The language with uncontrolled side-effects could of course provide both versions of that function, in which case you might want to give them different names, or at least different type signatures.

Which, in my Java implementation, they would have.