<html>
<title> Feeding a Large-scale Physics Application to Python </title>
<body bgcolor="#ffffff">

<h1> Feeding a Large-scale Physics Application to Python </h1>

David M. Beazley <br>
Department of Computer Science <br>
University of Utah <br>
Salt Lake City, Utah  84112 <br>
<tt> beazley@cs.utah.edu </tt> <br>
<p>
Peter S. Lomdahl <br>
Theoretical Division <br>
Los Alamos National Laboratory <br>
Los Alamos, New Mexico   87545 <br>
<tt> pxl@lanl.gov </tt> <br>

<p>
(Presented at the 6th International Python Conference, San Jose, California. October 14-17, 1997).

<h2> Abstract </h2>

<em>
We describe our experiences using Python with the SPaSM molecular
dynamics code at Los Alamos National Laboratory.  Originally developed
as a large monolithic application for massively parallel processing
systems, we have used Python to transform our application into a
flexible, highly modular, and extremely powerful system for performing
simulation, data analysis, and visualization.  In addition, we
describe how Python has solved a number of important problems related
to the development, debugging, deployment, and maintenance of
scientific software. </em>

<h2> Background </h2>

For the past 5 years, we have been developing a large-scale physics
code for performing molecular-dynamics simulations of
materials.  This code, SPaSM (Scalable Parallel Short-range
Molecular-dynamics), was originally developed for the Connection
Machine 5 massively parallel supercomputing system and later moved to
a number of other machines including the Cray T3D, multiprocessor Sun
and SGI systems, and Unix workstations [1,2].  Our goal has
been to investigate material properties such as fracture, crack
propagation, dislocation generation, friction, and ductile-brittle transitions [3].
The SPaSM code
helps us investigate these problems by performing atomistic
simulations--that is, we simulate the dynamics of every atom in a
material and hope to make sense of what happens.  Of course, the
underlying physics is not so important for this discussion.

<p>
While the development of SPaSM is an ongoing effort, we
have been hampered by a number of serious problems.
First, typical
simulations generate tens to hundreds of gigabytes of data that must
be analyzed. This task is not easily performed on a user's
workstation, nor is it economically feasible to buy everyone their own
personal desktop supercomputer.  A second problem is that of
interactivity and control.  We are constantly making changes to
investigate new physical models, different materials, and so forth.
This would usually require changes to the underlying C
code--a process that was tedious and not very user friendly. We
wanted a more flexible mechanism.
Finally, there were many difficulties associated with the
development and maintenance of our software.   While we are only a small
group, it was not uncommon for different users to have their 
own private copies of the software that had been modified in some manner. 
This, in turn, led to a maintenance nightmare that made it almost impossible
to update the software or apply bug-fixes in a consistent manner.

<p>
To address these problems, we started investigating the use of scripting
languages.  In 1995, we wrote a special purpose parallel-scripting language,
but replaced it with Python a year later (although the interface
generation tool for that scripting language lives on as SWIG).  In this
paper, we hope to describe some of our experiences with Python and how
it has helped us solve practical scientific computing problems.   In particular,
we describe the organization of our system, module building process,
interesting tools that Python has helped us develop, and why we think this
approach is particularly well-suited for scientific computing research.

<h2> Why Python? </h2>

Although our code originally used a custom scripting language, we decided
to switch to Python for a number of reasons :

<p>
<ul>
<li> Features.  Python had a rich set of datatypes, support for
object oriented programming, namespaces, exceptions, dynamic loading,
and a large number of useful modules.
<li> Syntax.  Our system is controlled through
text-based commands and scripts.  We felt that Python had a nice
syntax that does not require users to
type weird symbols ($,%,@, etc...) or use syntax that is radically
different than C.
<li> A small core.  The modular structure of Python makes it easy for us
to remove or add modules as needed.   Given the difficult task of running
on parallel machines and supercomputers, we felt that this structure
would make it easier for us to port Python to a variety of special-purpose
machines (since we could just remove problematic modules).
<li> Availability of documentation.   Users could go to the bookstore 
to find out more about Python.
<li> Support and stability.  Python appeared to be highly stable and
well supported through newsgroups, special interest groups, and the PSA.
<li> Freely available.  We have been able to
use and modify Python as needed for our application.  This would have
been impossible without access to the Python source.
</ul>

<p>
Another factor, not to be overlooked, is the increasing acceptance of
Python elsewhere in the computational science community.  Efforts at
Lawrence Livermore National Laboratory and elsewhere were
attractive--in particular, we saw Python as being a potential vehicle
for sharing modules and utilizing third-party tools [4,9,10].

<h2> System Organization </h2>

Our goal was to build a highly modular system that could perform
simulation, data analysis, and visualization--often performing
all of these tasks simultaneously.  Unfortunately, there is a tendency
to do this by building a tightly integrated
monolithic package (perhaps using a well-structured C++ class
hierarchy for example).  In our view, this is too
formal and restrictive.  We wanted to support modules that might only
be loosely related to each other.  For example, there is no need for a
graphics library to depend on the same structures as a simulation code
or to even be written in the same language for that matter.  Likewise,
we wanted to exploit third-party modules where no assumptions 
could be made about their internal structure.

<p>
The major components of our system take the form of C libraries.  When
Python is used, the libraries are compiled into shared libraries and
dynamically loaded into Python as extension modules.  The
functionality of each library is exposed as a collection of Python
"commands."  Because of this, the C and Python programming
environments are closely related.  In many cases, it is possible to
implement the same code in both C or Python. For example :

<blockquote>
<tt><pre>
/* A simple function written in C */
#include "SPaSM.h"
void run(int nsteps, double Dt, int freq) {
    int i;
    char filename[64];
    for (i = 0; i < nsteps; i++) {
         integrate_adv_coord(Dt);
         boundary_periodic();
         redistribute();
         force_eam();
         integrate_adv_velocity(Dt);
         if ((i % freq) == 0) {
            sprintf(filename,"Dat%d",i);
            output_particles(filename);
         }
    }
}</pre>
</tt>
</blockquote>
        
Now, in Python :

<blockquote><tt><pre>
# A function written in Python
from SPaSM import *
def run(nsteps, Dt, freq):
    for i in xrange(0, nsteps):
         integrate_adv_coord(Dt)
         boundary_periodic()
         redistribute()
         force_eam()
         integrate_adv_velocity(Dt)
         if (i % freq) == 0) :
            output_particles('Dat'+str(i))
</pre>
</tt>
</blockquote>

While C libraries provide much of the underlying functionality, the real power
of the system comes in the form of modules and scripts written entirely in Python.
Users write scripts to set up and control simulations.  Several major
components such as the visualization and data-analysis system make heavy
use of Python.  We also utilize a variety of modules in the Python
library and have dynamically loadable versions of Tkinter and the
Python Imaging Library [11].

<h2> Embedding Python and Hiding System Dependencies </h2>

One of the biggest implementation problems we have encountered is the
fact that the SPaSM code is a parallel application that relies heavily
upon the proper implementation of low-level system services such as
I/O and process management. Currently, it is possible to run the code
in two different configurations--one that uses message passing via the
MPI library and another using Solaris threads.  Using Python with
both versions of code requires a degree of care--in particular, we
have found it to be necessary to provide Python with enhanced I/O
support to run properly in parallel.  This work has been described
elsewhere [5,6].

<p>
Handling two operational modes introduces a number of problems related
to code maintenance and installation.  Traditionally, we would
recompile the entire system and all of its modules for each
configuration (placing the files in an architecture dependent
subdirectory).  Users would have to decide which system they wanted to
use and compile all of their modules by linking against the
appropriate libraries and setting the right compile-time options.
Changing configurations would typically require a complete recompile.

<p>
With dynamic loading and shared libraries however, we have been able
to devise a different approach to this problem.  Rather
than recompiling everything for each configuration, we use an
implementation independent layer of system wrappers.  These wrappers
provide a generic implementation of message passing, parallel I/O, and
thread management.  All of the core modules are then compiled
using these generic wrappers.   This makes the modules independent
of the underlying operational mode---to use MPI or threads,  we simply
need to supply a different implementation of the  system wrapper libraries.
This is easily accomplished by building two different versions of
Python that are linked against the appropriate system libraries. 
To run in a particular mode, we now just run the appropriate
version of Python (i.e. 'python' or 'pythonmpi').  The neat 
part about this approach is that all of the modules work with both
operational modes without any recompilation or reconfiguration.  If a
user is using threads, but wants to switch to MPI, they simply run
a different version of Python--no recompilation of modules is necessary.

<p>
A full discussion of writing system-wrappers can be found elsewhere.
In particular, a discussion of writing parallel I/O wrappers for
Python can be found in [5].  An earlier discussion of the technique we
have used for writing message passing and I/O wrappers can also be
found in [7].

<h2> Module Building with SWIG </h2>

To build modules, we have been using SWIG [8].
Each module is described by a SWIG interface file containing the ANSI
C declarations of functions, structures, and variables in that module.
For example :

<blockquote><pre>
// SWIG interface file
%module SPaSM
%{
#include "SPaSM.h"
%}
void integrate_adv_coord(double Dt);
void boundary_periodic();
void redistribute();
void force_eam();
void integrate_adv_velocity(double Dt);
int  output_particles(char *filename);
</pre></blockquote>

SWIG provides a logical mapping of the underlying C implementation into
Python.   During compilation, interface files are automatically
converted into wrapper code and compiled into Python modules.  This 
process is entirely transparent--changes made to the interface
are automatically propagated to Python whenever a module is recompiled.
Given the constantly evolving nature of research applications, this makes
it easy to extend and maintain the system.  

<h3> Separation of Implementation and Interface </h3>

An important aspect of SWIG is that it is requires no modifications to
existing C code which allows us to maintain a strict
separation between the implementation of C modules and their
Python interface.  We believe that this results in code that
is more organized and generally reusable.  There is no particular
reason why a C module should depend on Python (one might
want to use it as a stand-alone package or in a different application).
Despite using Python extensively, the SPaSM code can still be
compiled with no Python interface (of course, you lose all of
the benefits gained by having Python).  
We feel that most physics codes tend
to have a rather long life-span.  Maintaining a separation of
implementation and interface helps insure that our physics code will
be usable in the future--even if there are drastic changes in the
interface along the way.

<h3> Providing Access to Data Structures </h3>

For the purposes of debugging and data exploration, we have 
used SWIG to provide wrappers around C data structures.   For example,
a collection of C structures such as the following

<blockquote><pre>
typedef struct {
     double x,y,z;
} Vector;

typedef struct {
     int    type;
     Vector r;
     Vector v;
     Vector f;
} Particle;
</pre></blockquote>

can be turned into Python wrapper classes.  In addition, SWIG can extend structures with member functions as follows :

<blockquote> <pre>
// SWIG interface for vector and particle structures
%include datatypes.h

// Extend data structures with a few useful methods
%addmethods Vector {
     char *__str__() {
         static char a[1024];
         sprintf(a,"[ %0.10f, %0.10f, %0.10f ]", self->x, self->y, self->z);
         return a;
     }
}
%addmethods Particle {
     Particle *__getitem__(int index) {
          return self+index;
     }
     char *__str__() {
          // print out a particle
          ...
     }
}
</pre>
</blockquote>

When the Python interface is built, C structures now appear like
Python objects.  By providing Python-specific methods (such as
<tt>__getitem__</tt>) we can even provide array access.
For example, the following Python code would print out
all of the coordinates of stored particles :

<blockquote>
<pre>
# Print out all coordinates to a file
f = open("part.data","w")
p = SPaSM_first_particle()          # Get first particle pointer
for i in xrange(0,SPaSM_count_particles()):
    f.write("%f, %f, %f\n" % (p[i].r.x, p[i].r.y, p[i].r.z))
f.close()
</pre>
</blockquote>

While only a simple example, having direct access to underlying data has
proven to be quite valuable since we view the internal representation 
of data, check values, and perform diagnostics. 

<h3> Improving the Reliability of Modules </h3>

When instrumenting our original physics application to use scripting,
we found that many parts of the code were not written in an entirely
"reliable" manner.  In particular, the code had never been operated in
an event-driven manner.  Functions often made assumptions about
initializations and rarely checked the validity of input parameters.
To address these issues, most sections of code were gradually changed
to provide some kind of validation of input values and error recovery.
The SWIG compiler has also been extended to provide some of these
capabilities as well.
<p>
One of Python's most powerful features is its exception handling mechanism.
Exceptions are easily raised and handled in Python scripts as follows :

<blockquote><pre>
# A Python function that throws an exception
def allocate(nbytes):
	ptr = SPaSM_malloc(nbytes)
        if ptr == "NULL": raise MemoryError,"Out of memory!"

# A Python function that catches an exception
def foo():
	try:
		allocate(NBYTES)
	except:
		return    # Bailing out

</blockquote></pre>

We have borrowed this idea and implemented a similar exception handling
mechanism for our C code.  This is accomplished using functions in the
<tt>&lt;setjmp.h&gt;</tt> library and defining a few C macros for "<tt>Try</tt>", "<tt>Except</tt>",
"<tt>Throw</tt>", etc... Using these macros, many of our library functions now look like the following :

<blockquote><pre>
/* A C function that throws an exception */
void *SPaSM_malloc(size_t nbytes) {
    void *ptr = (void *) malloc(nbytes);
    if (!ptr) Throw("SPaSM_malloc : Out of memory!");
    return ptr;
}
</pre></blockquote>

Like Python, we allow C functions to
catch exceptions and provide their own recovery as follows :

<blockquote> <pre>
/* A C function catching an exception */
int foo() {
    void *p;
    Try {
        p = SPaSM_malloc(NBYTES);
    } Except {
        printf("Unable to allocate memory. Returning!\n");
        return -1;
    }
}
</pre></blockquote>

In the case of a stand-alone C application, exceptions can
be caught and handled internally.  Should an uncaught exception
occur, the code prints an error message and terminates. However,
when Python is used, we can 
generate Python exceptions using a SWIG user-defined
exception handler such as the following :

<blockquote> <pre>
%module SPaSM
// A SWIG user defined exception handler
%except(python) {
     Try {
          $function
     } Except {
          PyErr_SetString(PyExc_RuntimeError,SPaSM_error_msg());
     }
}
// C declarations 
...
</pre></blockquote>

The handler code gets placed into all of the Python
"wrapper" functions and is responsible for translating C exceptions
into Python exceptions.  Doing this makes our physics code operate
in a more seamless manner and gives it a precisely defined error recovery
procedure (i.e.  internal errors always result Python exceptions). For example :

<blockquote><pre>
>>> SPaSM_malloc(1000000000)
RuntimeError: SPaSM_malloc(1000000000). Out of memory!
(Line 52 in memory.c)
>>>
</pre></blockquote>

We have found error recovery to be critical.  Without it, simulations may 
continue to run, only to generate wrong answers or a mysterious
system crash.  

<h2> More Than Scripting </h2>

When we originally started using scripting languages, we thought 
they would mainly be a convenient mechanism for gluing C libraries
together and controlling them in an interactive manner.  However, 
we have come to realize that scripting languages are
much more powerful than this.  Now, we find ourselves
implementing significant functionality entirely in Python--bypassing
C altogether.   The most surprising (well, not really that surprising)
fact is that Python makes it possible to build very powerful tools with
only a small amount of extra programming.   In this section, we present
a brief overview of some of the tools we have developed--most of
which have been implemented largely in Python.

<h3> Object-Oriented Visualization and Data Analysis </h3>

Our original goal was to add a powerful data analysis and
visualization component to our application.  To this end, we developed
a lightweight high-performance graphics library implemented in C.
This library supports both 2D and 3D plotting and produces output in
the form of GIF images.  To make plots, a user writes simple C
functions such as the following :

<blockquote><pre>
void plot_spheres(Plot3D *p3, DataFunction func, double min, double max,
                  double radius) {
    Particle *p = Particles;
    int       i, npart, color;
    npart = SPaSM_count_particles();
    for (i = 0; i < npart; i++, p++) {
        /* Compute color value */
        value = (*func)(p);
        color = (value-min)/(max-min)*255;
        /* Plot it */
        Plot3D_sphere(p3,p->r.x,p->r.y,p->r.z,radius,color);
    }
}
</pre></blockquote>
When executed, this function produces a raw image such as the following
showing stacking-faults generated by a passing shock wave in an fcc crystal
of 10 million atoms :
<p>
<center>
<img src="bigshockraw.gif">
</center>
<p>
While simple, we often want our images to contain more information
including titles, axis labels, colorbars, time-stamps, and bounding
boxes.   To do this, we have built an object-oriented visualization
framework in Python.   Python classes provide methods for graph annotation
and common graph operations.   If a user wants to make a new kind of plot,
they simply inherit from an appropriate base class and provide a function
to plot the desired data. For example, a "Sphere" plot using the above C
function would look like this :

<blockquote><pre>
class Spheres(Image3D):
      def __init__(self, func, min, max, radius=0.5):
             Image3D.__init__(self, ... 
             self.func = func
             self.min  = min
             self.max  = max
             self.radius = radius
      def draw(self):
             self.newplot()
             plot_spheres(self.p3,self.func,self.min,self.max,self.radius)
</pre></blockquote>

To use the new image, one simply creates an object of that type and
manipulates it.  For example :

<blockquote><pre>
>>> s = Spheres(PE,-8,-3.5)   # Create a new image
>>> s.rotd(45)                # Rotate it down
>>> s.zoom(200)               # Zoom in
>>> s.show()                  # generate Image and display it
>>> ...
</pre></blockquote>

<center>
<img src="bigshock.gif">
</center>

<p>
Thus, with a simple C function and a simple Python class, we get a lot
of functionality for free, including methods for image manipulation,
graph annotation, and display.  In the current system, there are about
a dozen different types of images for 2D and 3D plotting.  It is also
possible to perform filtering, apply clipping planes, and other
advanced operations.    Most of this is supported by about 2000 lines of
Python code and a relatively small C library containing performance
critical operations.

<h3> Web-Based Simulation Monitoring </h3>

One feature of many physics codes is that they tend to run for a long
time.  In our case, a simulation may run for tens to hundreds of
hours.  During the course of a simulation, it is desirable to check on
its status and see how it is progressing.  Given the strong Internet
support already bundled with Python, we decided to write a simple
physics web-server that could be used for this purpose.  Unlike a
more traditional server, our server is used by "registering" various
objects that one wants to look at during the course of a simulation.
The simulation code then periodically polls a network socket to see if
anyone has requested anything.  If so, the simulation will stop for a
moment, generate the requested information, send it to the user, and
then continue on with the calculation.  A simplified example of using
the server is as follows :

<blockquote> <pre>
# Simplified script using a web-server
from vis import *
from web import *

web = SPaSMWeb()
web.add(WebMain(""))
web.add(WebFileText("Msg"+`run_no`))

# Create an image object

set_graphics_mode(HTTP)
ke = Spheres(KE,0,20)
ke.title = "Kinetic Energy"
web.add(WebImage("ke.gif",ke))

def run(nsteps):
	for i in xrange(0,nsteps):
		integrate(1)           # Integrate 1 timestep
                web.poll()             # Anyone looking for me?

# Run it
run(10000)
</pre></blockquote>

While simple, this example gives the basic idea. We create a
server object and register a few "links."  When a user connects with
the server, they will be presented with some status information and a
list of available links.  When the links are accessed, the server
feeds real-time data back to the user.  In the case of
images, they are generated immediately and sent back in GIF format. During
this process, no temporary files are created, nor is the server
transmitting previously stored information (i.e. everything is 
created on-the-fly). 

<p>
While we are still refining the implementation, this approach has turned
out to be useful--not only can we periodically check up
on a running simulation, this can can be done from any machine that
has a Web-browser, including PCs and Macs running over a modem line
(allowing a bored physicist to check on long-running jobs from home
or while on travel).  Of course, the most amazing fact of all is that
this was implemented by one person in an afternoon and only involved
about 150 lines of Python code (with generous help from the Python library).

<h3> Development Support </h3>

Finally, we have found Python to be quite useful for supporting
the future development of our code.  Some of this comes in the form of
sophisticated debugging--Python can be used to analyze internal data
structures and track down problems.  Since Python can read
and modify most of the core C data structures, it is possible to
prototype new functions or to perform one-time operations without ever
loading up the C compiler.

<p>
We have also used Python in conjunction with code
management. One of the newer problems we have faced is the task of
finding the definitions of C functions (for example, 
we might want to know how a particular command has been
implemented in C).  To support this, we have written tools for
browsing source directories, displaying definitions, and spawning
editors.  All of this is implemented in Python and can be performed
directly from the Physics application.  Python has made these kinds
of tools very easy to write--often only requiring a few hours of
effort.

<h2> Python and the Development of Scientific Software </h2>

The adoption of Python has had a profound effect on the overall
structure of our application.  With time, the code became more
modular, more reliable, and better organized.   
Furthermore, these gains have been achieved
without significant losses in performance, increased coding complexity,
or substantially increased development cost.   It should also be stressed
that automated tools such as SWIG have played a critical role in this
effort by hiding the Python-C interface and allowing us to concentrate
on the real problems at hand (i.e. physics).  

<p>
One point, that we would like to strongly emphasize, is the dynamic
nature of many scientific applications.  Our application is not a huge
monolithic package that never changes and which no one is supposed to
modify.  Rather, the code is <em>constantly</em> changing to explore
new problems, try new numerical methods, or to provide better
performance.  The highly modular nature of Python works great in
this environment.  We provide a common repository of modules that are shared by all
of the users of the system.  Different users are typically responsible
for different modules and all users may be working on different modules
at the same time. 
Whenever changes to a module are made, they are immediately propagated
to everyone else.  In the case of bug-fixes, this means that patched
versions of code are available immediately.  Likewise, new
features show up automatically and can be easily tested by everyone.
And of course, if something breaks--others tend to notice almost immediately
(which we feel is a good thing).

<h2> Limitations of Python </h2>

While we consider our Python "experiment" to be highly successful, we
have experienced a number of problems along the way.  The first is
merely a technical issue--it would be nice if the mechanism for
building static extensions to Python were made easier.  While we are
currently using dynamic loading, many machines, especially
supercomputing systems, do not support it.  On these machines it is
necessary to staticly link everything.  Libraries help simplify the
task, but combining different statically linked modules together into
a single Python interpreter still remains magical and somewhat
problematic (especially since we are creating an application that
changes a lot--not merely trying to patch the Python core with a new
extension module).

<p>
A second problem is that of education--we have found that some users
have a very difficult time making sense of the system (even we're
somewhat amazed that it really works).
Interestingly enough, this does not seem to be directly related to the
use of C code or Python for that matter, but to issues related to the
configuration, compilation, debugging, and installation of modules.  While most
users have written programs before, they have never dealt with shared
libraries, automated code generators, high-level languages, and third-party packages
such as Python.  In other cases, there seems to be a
"fear of autoconf"--if a program requires a configure script, it must
be too complicated to understand (which is not the case
here).  The combination of these
factors has led some users to believe that the system is far more
complicated and fragile than it really is.  We're not sure how to combat
this problem other than to try and educate the users
(i.e. it's essentially the same code as before, but with a really cool
hack).

<h2> Conclusions and Future Work </h2>

Python is cool and we plan to keep using it. However much work
remains.  One of the more exciting aspects of Python is its solid
support for modular programming and the culture of cooperation in the
Python community. There are already many Python-related scientific
computing projects underway.  If these efforts were coordinated 
in some manner, we feel that the results could be spectacular.

<h2> Acknowledgments </h2>

We would like acknowledge our collaborators Brad Holian, Tim Germann, Shujia Zhou, and Wanshu Huang at Los Alamos National Laboratory.  We would also like
to acknowledge Paul Dubois and Brian Yang at Lawrence Livermore
National Laboratory for many interesting conversations concerning Python
and its use in physics applications.  We would also like to
acknowledge the Scientific Computing and Imaging Group at the
University of Utah for their continued support.  Finally, we'd like to
thank the entire Python development community for making a truly
awesome tool for solving real problems.  Development of the SPaSM code
has been performed under the auspices of the United States Department
of Energy.

<h2> References </h2>

<p>
[1] D.M.Beazley and P.S. Lomdahl, "Message
Passing Multi-Cell Molecular Dynamics on the Connection Machine 5,"
Parallel Computing 20 (1994), p. 173-195. <p>

<p>
[2] P.S.Lomdahl, P.Tamayo, N.Gronbech-Jensen,
and D.M.Beazley, "50 Gflops Molecular Dynamics on the CM-5," Proceedings
of Supercomputing 93, IEEE Computer Society (1993), p.520-527. <p>

<p>
[3] S.J.Zhou, D.M. Beazley, P.S. Lomdahl, B.L. Holian, Physical Review
Letters. 78, 479 (1997).

<p>
[4] P. Dubois, K. Hinsen, and J. Hugunin, Computers in Physics (10), (1996)
p. 262-267.

<p>
[5] D. M. Beazley and P.S. Lomdahl, "Extensible Message Passing Application
Development and Debugging with Python", Proceedings of IPPS'97, Geneva
Switzerland, IEEE Computer Society (1997).

<p>
[6] T.-Y.B. Yang, D.M. Beazley, P. F. Dubois, G. Furnish, "Steering Object-oriented Computations with
Python", Python Workshop 5, Washington D.C., Nov. 4-5, 1996.

<p>
[7] D.M. Beazley and P.S. Lomdahl, "High
Performance Molecular Dynamics Modeling with SPaSM : Performance and
Portability Issues," Proceedings of the Workshop on Debugging and
Tuning for Parallel Computer Systems, Chatham, MA, 1994 (IEEE Computer
Society Press, Los Alamitos, CA, 1996), pp. 337-351.

<p>
[8] D.M. Beazley, "Using SWIG to Control, Prototype, and Debug C Programs with Python",
Proceedings of the 4th International Python Conference, Lawrence Livermore 
National Laboratory, June 3-6, 1996. 

<p>
[9] K. Hinsen, The Molecular Modeling Toolkit, <tt>http://starship.skyport.net/crew/hinsen/mmtk.html</tt>

<p>
[10] Jim Hugunin, Numeric Python, <tt>http://www.sls.lcs.mit.edu/jjh/numpy</tt>

<p>
[11] Fredrik Lundh, Python Imaging Library, <tt>http://www.python.org/sigs/image-sig/Imaging.html</tt>
</body>
</html>