> Subject: Re: External behaviors (was Re: MISC: Inlined Sound Support)
> > Yes, this is the crux of the issue. We don't really need to specify
> > anything about the scripting language itself, it's the binding that
> > is important. And as you alluded to before, this "binding" should be the
>
> Explain this to me. I write my script in Cobra, you write yours
> in C, someone else writes theirs in Prolog. How are all these different
> scripts supposed to be executed by the browsers? API aside. Let's say
> we define an API, well then when I write a script and then "compile" it
> for Windows, Mac, SGI, SUN, etc... Am I missing something here?
First, assume that all of the browsers implement some form of "external"
interface. This interface can be used to hook up an interaction engine to the
browser. The browser and the interaction engine(s) are seperate peices. You
might have one browser and several interaction engines hooked up to it. The
rationale behind this is that not every language is suited to all tasks, and
that the problems of languages and networking are very different from those of
rendering. Asume for a moment (for simplicity) that we are ignoring the case of
multiple different types of interaction engines running concurrently (!).
Compilation/interpretation of the scripts occurs at the "client" - on the same
machine which is running the browser, not at the "server". We guarentee similar
behaviors by transmitting source, as we do with VRML, rather than bytecodes or
other similar stuff (though there is nothing inherent in this approach which
precludes optimisation by doing this later on). We're already taking the hit of
running stuff thru a parser for the scene descrption language- to acheive
similar goals for scene descriptions. We have an obviously large overhead on
scene startup, but could get something working quite quickly in this direction,
then move to a bytecode behavior distribution later, when we have languages
which do cross platform interpretation. Kaleida lab's scriptX is one example of
such a system; Java is another. What Kevin is alluding to (ie, multiple versions
compiled for different platforms) above has a lot of security problems, unless
it it "compiled" into bytecodes which run in an interpreter we know to be
secure (Java is one example of such a system).
Behaviors are expressed using text or other information stored in URLs (UR*s :)
), with MIME types(s) for different language types. When an external interaction
engine is hooked up to a browser, the browser "knows" that it is capable of
processing the type(s) of interaction behaviors supported by the engine, and it
also knows that when it receives behaviors of this type, it should give them,
via the interface, to the interaction engine, in a similar manner to the way
that external "helper" applications currently act. Not that what the external
helper gets does not have to be code, it could be anything. Patterns,
communications information, etc.
The interface mechanism between browser and external helpers does **not** have
to be the same on every platform (right now, it could not easily be !). However,
the way in which the interface **works** is common across platforms, in that it
each implementation allows external applications to make changes to the scene
graph and also to receive "events" from the browser (which are primarily
generated by user interactions. An example of such an event might be a user
selecting a particular object. I'm sure there is plenty of research in this
direction which can be consulted to find out how to specify which interactions
we are interested in). We add features to the scene description language (VRML)
to specify which objects are "hot" to events, to make sure we aren't overloading
the interface between the browser and the external applications we unneeded
crap. We can either use the standard inventoresque stuff here, or define
another, simple, mechanism, which tells the browser "I am interested in this
object, or "when this pattern is drawn using the "cursor", tell me. I have not
looked into this in enough detail to provide a detailed proposal - mainly due to
workload in other areas. This mechanism is similar to conventional callbacks in
event driven oo-systems, but here it results in a message sent to registered
external applications via the interface. We should provide facilities for events
to be designated as asynchronous and synchronous (this will be handy later on).
When an external application receives a message which tells it than an event has
occurred, it consults some form of map which is specified in the behaviour
information downloaded from the URL. This map relates events generated by the
browser to behaviors. The behavior is then executed. Behaviors can (via language
specific hookups to the interface) affect the scene graph, and probably cause
other events to occur in the browser. Behaviors are executed on a peicemeal
basis as a result of events generated in the browser.
In this way of thinking the browser acts as the focal point for both user and
code interactions. User and machine interact through a common space. My one
concern with all of this is latency and download times. We already have parsing
going on, and only time will tell if this is reasonable approach to take. I do
know this tho, a binary cross-platform scene description AND interaction
langauge would be a tough row to hoe right now - requing a large, tight knit
team. I think VRML would be better served by doing a framework, experimenting in
hooking up existing languages, and modifying one of them (with available code)
to suit our needs, or by using something secure and existant, like Java, if it
fits.
One final point. It isn't only behaviors which would fit into this framework.
There are a lot of other things which would take the place of a simple
communications framework - a MUD interface, for example, or something new and
different, entirely.
That's it for now. Phew !
-- Mike