Google Git
Sign in
fuchsia / third_party / swig / 84c385ef832f9ec0d69119c39331b988a314b38a / . / swigweb / papers / Perl98 / swigperl.htm
blob: 03c9ce88c918d7bb13e9ee1fba91ee617f2e9308 [file] [log] [blame]
<html>
<title> Perl Extension Building with SWIG </title>
<body bgcolor="#ffffff">
<h1> Perl Extension Building with SWIG</h1>
(Presented at the O'Reilly Perl Conference 2.0, August 17-20, 1998, San Jose, California.)
<p>
<b> David M. Beazley </b> <br>
<em> Dept. of Computer Science <br>
University of Chicago <br>
Chicago, IL 60637 <br>
</em>
<p>
<b> David Fletcher </b> <br>
<em>Fusion MicroMedia, Corp. <br>
Longmont, CO 80501 <br>
</em>
<p>
<b>Dominique Dumont</b> <br>
<em> Hewlett Packard <br>
Lab TID <br>
5 Ave Raymond Chanas <br>
38053 Grenoble cedex <br>
France <br> </em>
<p>
[ <a href="swigperl.pdf">PDF </a> ]
<h2> Abstract </h2>
<em>
SWIG (Simplified Wrapper and Interface Generator) is a freely available
tool that integrates Perl, Python, Tcl, and other scripting languages
with programs written in C, C++, and Objective-C. This paper
provides an introduction to SWIG and shows how it can be used to
construct Perl extension modules. In addition, a number of applications
in which SWIG has been utilized are described. While SWIG is similar
to other Perl extension building tools such as xsubpp and h2xs, SWIG
has a number of unique features that help simplify the task of creating
Perl extension modules. Many of these features are described as well as
limitations and future directions.
This paper is
primarily intended for developers who are interested in combining Perl
with applications written in C or C++ as well as current SWIG users
who are interested in learning more about some of SWIG's advanced features.
</em>
<h2> 1 Introduction </h2>
One of Perl's greatest strengths is its ability to simplify hard
programming tasks as well as being able to solve the odd and varied
computing problems that occur on a day-to-day basis. While it would
be nice to use Perl (or other high-level languages) for everything,
this is simply not practical for many applications. In fact,
performance critical tasks, low-level systems programming, and
complex data structures are likely to be implemented
in a compiled language such as C or C++ (and may be easier to manage
in such languages). Furthermore, developers often need to work with a
wide variety of existing applications and ``legacy'' systems that
are written in such languages.
<p>
The integration of Perl and code written in compiled languages has a
number of practical benefits. First, it allows existing C/C++
applications to be incorporated into a high-level interpreted
environment. This environment provides greater flexibility and often
simplifies development since debugging and testing can be performed
using Perl scripts. Second, Perl can serve as a powerful user
interface. In other words, rather than writing a user interface
from scratch, it is possible to use a Perl interpreter instead.
This also allows other for other possibilities such as graphical
user interface development with Perl/Tk. Finally, Perl
provides developers with a mechanism for assembling and controlling
software components. Rather than creating a huge monolithic package,
C/C++ programs can be packaged as collections of Perl extension
modules. As a result, programs become more modular and easier to
maintain. Furthermore, it is even possible to combine entirely
different programs together within a shared Perl interpreter.
<p>
This paper provides an introduction and overview of SWIG, a tool
designed to integrate C code with a variety of scripting
languages including Perl, Python, and Tcl. Currently, SWIG can
construct Perl extension modules on Unix and Windows-NT systems. It also
supports the ActiveState Perl for Windows and Perl4. SWIG has been
freely available since February, 1996 and has been
previously described in <em>Advanced Perl Programming</em>, <em>The
Perl Journal</em>, and <em>Dr. Dobb's Journal</em>[1,2,3].
In addition, SWIG is packaged with a 300 page user manual describing its
use [4]. The goal of this paper is not to repeat all of this information, but to provide an overview of
SWIG, demonstrate the use of some of its more advanced features, and describe
some of the ways that it is currently being used. The authors include the developer
of SWIG and two of SWIG's foremost Perl experts who have made substantial
contributions to SWIG's development.
<h2> 2 Perl Extension Building </h2>
To interface Perl with code written in C or C++, it is
necessary to write wrappers that serve as a glue layer between
the Perl interpreter and the underlying C code. These wrappers
are responsible for converting data between Perl and C, reporting
errors, and other tasks. Perl is packaged with several tools
for creating these wrappers. One such tool is <tt>xsubpp</tt>, a
compiler that takes interface definitions written in a special
language known as XS and converts them into wrappers. For example,
suppose that you had the following C function:
<blockquote><pre>
int fact(int n);
</pre></blockquote>
To wrap this function into a Perl module with <tt>xsubpp</tt>, you would write
the following XS file:
<blockquote><pre>
/* file : example.xs */
extern int fact(int n);
MODULE = Example PACKAGE = Example
int
fact(n)
int n
</pre></blockquote>
When processed with <tt>xsubpp</tt>, the following wrapper file is produced
<blockquote><pre>
#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"
extern int fact(int n);
XS(XS_Example_fact)
{
dXSARGS;
if (items != 1)
croak("Usage: Example::fact(n)");
{
int n = (int)SvIV(ST(0));
int RETVAL;
RETVAL = fact(n);
ST(0) = sv_newmortal();
sv_setiv(ST(0), (IV)RETVAL);
}
XSRETURN(1);
}
XS(boot_Example)
{
dXSARGS;
char* file = __FILE__;
XS_VERSION_BOOTCHECK ;
newXS("Example::fact",
XS_Example_fact, file);
ST(0) = &sv_yes;
XSRETURN(1);
}
</pre></blockquote>
To use the module, the wrapper code must be compiled and linked into a
shared library that can be dynamically loaded into the Perl interpreter.
The easiest way to do this is with the MakeMaker utility by writing a
script as follows:
<blockquote><pre>
# file : Makefile.PL
use ExtUtils::MakeMaker;
WriteMakefile(
'NAME' =&gt; 'Example',
'OBJECT' =&gt; 'example.o fact.o'
);
</pre></blockquote>
this script is then used to create a Makefile and module as follows:
<blockquote><pre>
unix > perl Makefile.PL
unix > make
unix > make install
</pre></blockquote>
Finally, in addition to creating the C component of the extension module,
it is necessary to write a <tt>.pm</tt> file that is used to load
and initialize the module. For example,
<blockquote><pre>
# file : Example.pm
package Example;
require Exporter;
require DynaLoader;
@ISA = qw(Exporter DynaLoader);
bootstrap Example;
1;
</pre></blockquote>
At this point, you should have a working Perl extension module. In
principle, building a Perl extension requires an XS specification for
every C function that is to be accessed. To simplify the process of
creating these specifications, Perl includes <tt>h2xs</tt>, a tool that
converts C header files to XS descriptions. While useful, <tt>h2xs</tt>
is somewhat limited in its ability to handle global variables,
structures, classes, and more advanced C/C++ features. As a result,
<tt>h2xs</tt> can be somewhat difficult to use with more complex
applications.
<h2> 3 SWIG Overview </h2>
In a nutshell, SWIG is a specialized compiler that transforms ANSI
C/C++ declarations into scripting language extension wrappers.
While
somewhat similar to <tt>h2xs</tt>, SWIG has a number of
notable differences. First, SWIG is much less internals oriented than
XS. In other words, SWIG interfaces can usually be constructed
without any knowledge of Perl's internal operation. Second, SWIG is designed to
be extensible and general purpose. Currently, wrappers can be
generated for Perl, Python, Tcl, and Guile. In addition, experimental
modules for MATLAB and Java have been developed. Finally, SWIG
supports a larger subset of C and C++ including structures,
classes, global variables, and inheritance. This section provides a tour of SWIG and describes many
of its interesting features.
<h3> 3.1 A Small Taste</h3>
As a first example, suppose that you wanted to build a Perl
interface to Thomas Boutell's gd graphics library [footnote:
The gd library is a freely available graphics library for
producing GIF images and can be obtained at <tt>
<a href="http://www.boutell.com/gd/gd.html">
http://www.boutell.com/gd/gd.html</a></tt>. A Perl module to gd,
developed by Lincoln Stein, is also available on CPAN so
interested readers are encouraged to compare the results of
using SWIG against an existing Perl extension.]
<p>
Since gd is a C library,
images are normally created by writing C code such as
follows:
<blockquote><pre>
#include "gd.h"
int main() {
gdImagePtr im;
FILE *out;
int blk,wht;
/* Create an image */
im=gdImageCreate(200,200);
/* Allocate some colors */
blk=gdImageColorAllocate(im,0,0,0);
wht=gdImageColorAllocate(im,255,255,255);
/* Draw a line */
gdImageLine(im,20,50,180,140,wht);
/* Output the image */
out=fopen("test.gif","wb");
gdImageGif(im,out);
fclose(out);
/* Clean up */
gdImageDestroy(im);
}
</pre></blockquote>
By building a Perl interface to gd, our goal is to write similar code
in Perl. Thus, the functionality of the gd library must be exposed to
the Perl interpreter. To do this, a SWIG interface
file can be written as follows:
<blockquote><pre>
// File : gd.i
%module gd
%{
#include "gd.h"
%}
typedef gdImage *gdImagePtr;
gdImagePtr gdImageCreate(int sx, int sy);
void gdImageDestroy(gdImagePtr im);
void gdImageLine(gdImagePtr im,
int x1, int y1,
int x2, int y2,
int color);
int gdImageColorAllocate(gdImagePtr im,
int r, int g, int b);
void gdImageGif(gdImagePtr im, FILE *out);
// File I/O functions (explained shortly)
FILE *fopen(char *name, char *mode);
void fclose(FILE *);
</pre></blockquote>
In this file, the ANSI C prototypes for every function that we would
like to access from Perl are listed.
In addition, a number of SWIG
directives (which are always preceded by a ``%'') appear. The <tt>
%module</tt> directive specifies the name of the extension module. The
<tt>%{, %}</tt> block is used to insert literal code into the output
wrapper file [footnote : This syntax is derived from lex and yacc]. In
this case, we simply include the ``<tt>gd.h</tt>'' header file.
Finally, a few file
I/O functions also appear. While not part of gd, these functions
are needed to manufacture file handles used by several gd functions.
<p>
To run SWIG, the following command is executed:
<blockquote><pre>
unix &gt; swig -perl5 gd.i
Generating wrappers for Perl 5
</pre></blockquote>
This produces two files, <tt>gd_wrap.c</tt> and
<tt>gd.pm</tt>. The first file contains C wrapper
functions that appear similar to the output that
would have been generated by <tt>xsubpp</tt>.
The
<tt>.pm</tt> file contains supporting Perl code needed to
load and use the module.
<p>
To build the module,
the wrapper file is compiled and linked into a shared library.
This process varies on every machine (consult the man pages), but the following
steps are performed on Linux:
<blockquote><pre>
unix &gt; gcc -fpic -c gd_wrap.c \
-Dbool=char \
-I/usr/lib/perl5/i586-linux/5.004/CORE
unix &gt; gcc -shared gd_wrap.o -lgd -o gd.so
</pre></blockquote>
At this point, the module is ready to use. For example,
the earlier C program can be directly translated into the
following Perl script:
<blockquote><pre>
#!/usr/bin/perl
use gd;
# Create an image
$im = gd::gdImageCreate(200,200);
# Allocate some colors
$blk=gd::gdImageColorAllocate($im,0,0,0);
$wht=gd::gdImageColorAllocate($im,255,
255,255);
# Draw a line
gd::gdImageLine($im,20,50,180,140,$wht);
# Output the image
$out=gd::fopen("test.gif","wb");
gd::gdImageGif($im,$out);
gd::fclose($out);
# Clean up
gd::gdImageDestroy($im);
</pre></blockquote>
<h3> 3.2 Input Files</h3>
In the gd example, SWIG was given a special interface file
containing a list of the C declarations to be included
in the Perl module. When working with a large C library, interface
files can often be constructed by copying an
existing header file and modifying it slightly. However, in some
cases, it is possible to include a header file as follows:
<blockquote><pre>
%module
%{
#include "gd.h"
%}
// Grab the declarations from gd.h
%include "gd.h"
// Some file I/O functions
FILE *fopen(char *name, char *mode);
void fclose(FILE *);
</pre></blockquote>
The <tt>%include</tt> directive tells SWIG to include a file and
parse all of the declarations it contains. In this case, the
interface would now wrap every function in the gd library
as opposed to the half-dozen functions listed in the first example.
<p>
SWIG also includes a C preprocessor that can be used for macro
expansion and conditional compilation. If a new application is
being written with SWIG in mind, header files can be written as
follows:
<blockquote><pre>
#ifdef SWIG
%module gd
%{
#include "gd.h"
%}
#endif
/* C declarations */
...
</pre></blockquote>
With this approach, the file can serve as both a valid C header
file and as an interface specification. The <tt>SWIG</tt> symbol
is only defined when SWIG is parsing so special directives can be
easily hidden from the C compiler as needed.
<p>
Finally, for the truly lazy, SWIG can sometimes be run directly on
C header and source files. For example,
<blockquote><pre>
% swig -perl5 -module gd gd.h
% swig -perl5 -module example example.c
</pre></blockquote>
Most users, however, use a mix of dedicated interface files
and header files.
<h3> 3.3 Data Model</h3>
The most critical part of interfacing Perl to C programs is
the management of data. Since Perl and C utilize a different set of
internal datatypes, wrapper generators are responsible for producing
code that marshals data and objects between languages. For fundamental types
such as <tt>int</tt> and <tt>double</tt> the conversion process is
straightforward. However, pointers, arrays, structures, and objects
complicate the process. Furthermore, since most C/C++ programs
make extensive use of these datatypes, it is important for wrapper generators
to support as many of these datatypes as possible.
<h4> 3.3.1 Pointers</h4>
SWIG maps C pointers and C++ references into Perl blessed references. These references
contain both the value of the pointer itself, plus a type-signature.
In the gd example, pointers were used to manage both images
and files. If one were to print out the value a pointer, it would
appear as follows:
<blockquote><pre>
gdImagePtr=SCALAR(0x80b9914)
</pre></blockquote>
SWIG uses the type-signature to perform run-time checking of all pointer values. These checks emulate
many of the checks that would have been performed by a C compiler.
When an invalid Perl datatype
or pointer of invalid type is used, a run-time error is generated. For example,
<blockquote><pre>
% perl
use gd;
$f = gd::fopen("test.gif","w");
gd::gdImageLine($f,20,50,180,140,0);
Type error in argument 1 of gdImageLine.
Expected gdImagePtr. at - line 3.
</pre></blockquote>
Type-checking is based on the name of each datatype. However,
the type-checker also keeps track of C++ inheritance hierarchies and
<tt>typedef</tt> definitions. Thus, an acceptable pointer type includes any
alternate names that might have been created with a <tt>typedef</tt>
declaration as well as any derived datatypes in C++.
<p>
When pointers are manipulated in Perl, they are opaque values. That
is, pointers can be created and passed around to other C functions,
but they can not be dereferenced directly. Thus, in the example, it
is difficult (or impractical) for a user to directly manipulate the internal
representation of an image from the Perl interpreter. Furthermore,
SWIG, by default, handles all pointers in a uniform manner. Thus,
datatypes such as <tt>FILE *</tt> are represented as blessed references
even though such types may appear remarkably similar to other Perl
datatypes such as file handles.
<h4> 3.3.2 Arrays</h4>
SWIG maps all arrays into pointers where the ``value'' of an array
is simply a pointer to the first element in the array. This is
the same model used by C compilers and like C, SWIG performs no
bounds or size checking. Thus, a function such as
<blockquote><pre>
void foo(double a[4][4]);
</pre></blockquote>
would accept <em>any</em> object of type <tt>double *</tt>. It is
up to the user to ensure that the pointer is valid and that it
points to memory that has been properly allocated.
<h4> 3.3.3 Structures and Objects</h4>
Finally, all structures and objects are represented as
pointers. This includes cases where objects are manipulated
by value. For example, the functions
<blockquote><pre>
double dot_product(Vector a, Vector b);
Vector cross_product(Vector a, Vector b);
</pre></blockquote>
are transformed by SWIG into the following wrappers [footnote: When
C++ is used, SWIG uses the default copy constructor instead of
<tt>malloc</tt>]:
<blockquote><pre>
double
wrap_dot_product(Vector *a, Vector *b) {
return dot_product(*a,*b);
}
Vector *
wrap_cross_product(Vector *a, Vector *b) {
Vector *r;
r = (Vector *) malloc(sizeof(Vector));
*r = cross_product(*a,*b);
return r;
}
</pre></blockquote>
The representation of objects by reference avoids
the problem of marshaling objects between a C and Perl representation--a
process that would be extremely difficult for
very complicated C datatypes. It also provides better performance
since manipulating references is more efficient than copying object
data back and forth between languages. Finally, the use of references
closely matches the way in which most C/C++ programs already handle
objects.
<p>
The downside to this approach is that objects are opaque in Perl.
This prevents users from examining their contents directly. In
addition, SWIG wrappers occasionally need to perform implicit memory
allocations as shown above. It is up the user to free the resources
used by such functions (or learn to live with a memory leak). Of
course, this naturally brings us to the next topic.
<h4> 3.3.4 Memory Management</h4>
SWIG maintains a strict separation between the management of Perl and C
objects. While Perl uses reference counting to keep track of its
own objects, this scheme is not extended to C/C++
extensions created with SWIG. Thus, when Perl destroys a blessed
reference containing the value of a C pointer, only the pointer
value disappears, not the underlying C data that it points to.
<p>
From a user standpoint, SWIG generated C/C++ extensions follow the
same memory management rules as the underlying application.
Thus, if a program relies on <tt>malloc</tt> and <tt>free</tt> to allocate
and deallocate objects, these will also be used from the Perl
interpreter. Likewise, a C++ extension typically requires
explicit invocation of constructors and destructors. Furthermore, for
functions that implicitly allocate memory as in the previous section,
it is up to the user to explicitly destroy the result using <tt>
free</tt> or a C++ destructor. While such a scheme may seem problematic,
it is no less problematic than memory management in C (which may or
may not be a good thing depending on your point of view). Even if
it were possible to have Perl automatically manage C/C++ objects, this
would be an inherently dangerous affair--especially since Perl has no
way to know how an underlying C application really operates. Furthermore, it
would be a fatal error for Perl to deallocate objects that were still
in use.
Therefore, SWIG leaves memory management largely up the user.
<h4> 3.3.5 Pointers, Arrays, and Perl</h4>
A common confusion among some novice users is the difference between
C datatypes and similar Perl datatypes.
In particular, Perl references are not the
same as a C pointers and Perl arrays are not the same as C arrays. Differences
also apply to other datatypes such as files (this is the reason that
the simple example included prototypes for <tt>fopen</tt> and <tt>fclose</tt>).
The primary reason for
these differences is that objects in Perl have a different internal
representation than objects in C. For example, a Perl array is
represented as a collection of references to Perl objects which may be of mixed
types. The internal representation of this array is entirely
different than what would be used for a normal C array. Therefore, it
is impossible to take a Perl array and pass it in unmodified form to
an arbitrary C function.
<p>
The difference between Perl and C datatypes often arises with
C functions such as the following:
<blockquote><pre>
/* Plot some points */
void
plotpts(gdImagePtr im, int x[], int y[],
int npts, int c)
{
for (int i = 0; i &lt; npts; i++) {
gdImageSetPixel(im,x[i],y[i],c);
}
}
</pre></blockquote>
Ideally, a user might want to pass Perl arrays as arguments
as follows:
<blockquote><pre>
@a = (10,20,30,40);
@b = (50,70,60,200);
gd::plotpts($im,\@a,\@b,4,1); # Error!
</pre></blockquote>
However, this script generates a type error instead of acting
as one might expect. While such behavior may seem restrictive
or bizarre, SWIG has been deliberately designed to operate
in this manner. In fact, there are even benefits to this approach.
If Perl arrays were to be used as C arrays, a
copy would be made, verified for type-correctness, and deallocated every
time an array was passed to a C function. For large arrays, this would
introduce a substantial performance overhead. Space requirements are
also a concern for some C programs. For example, a numerical application
might manipulate arrays with millions of elements. Converting such
arrays to and from a Perl representation would clearly introduce substantial memory
and performance overhead. In contrast, manipulating pointers to such arrays
is easy and efficient.
<p>
It should also be noted that SWIG provides a variety of customization options
that can be used to change its behavior. In fact, one can even make SWIG map
Perl arrays into C arrays if desired. Therefore, most restrictions
can be eliminated with a little extra work. Some of these customization techniques are
described shortly.
<h3> 3.4 Helper Functions </h3>
Sometimes the Perl interface constructed by SWIG is lacking in
functionality or is difficult to use. For example, in the previous
section, a function operating on C arrays was presented. To construct
C arrays from Perl, it is necessary to add some additional
functions to the SWIG interface. This can be done using the
<tt>%inline</tt> directive as follows:
<blockquote><pre>
// Add some helper functions for C arrays
%inline %{
int *int_array(int size) {
return (int *) malloc(sizeof(int)*size);
}
void int_destroy(int *a) {
free(a);
}
void int_set(int *a, int i, int val) {
a[i] = val;
}
int int_get(int *a, int i) {
return a[i];
}
%}
</pre></blockquote>
When SWIG builds the scripting interface, these functions
become part of the extension module and can be used as follows:
<blockquote><pre>
# Convert a Perl array into a C int array
sub create_array {
$len = scalar(@_);
$ia = gd::int_array($len);
for ($i = 0; $i &lt; $len; $i++) {
val = shift;
gd::int_set($ia,$i,$val);
}
return $ia;
}
@a = (10,20,30,40);
@b = (50,70,60,200);
$ia = create_array(@a); # Create C arrays
$ib = create_array(@b);
gd::plotpts($im,$ia,$ib,4,1);
...
gd::int_destroy($ia);
gd::int_destroy($ib);
</pre></blockquote>
<h3> 3.5 Classes and Structures</h3>
While SWIG represents all objects as opaque pointers, the
contents of an object can be examined and modified through
the use of accessor functions as follows:
<blockquote><pre>
/* Extract data from an object */
double Point_x_get(Point *p) {
return p-&gt;x;
}
/* Invoke a C++ member function */
int Foo_bar(Foo *f) {
return f-&gt;bar();
}
</pre></blockquote>
From a Perl script, a user simply passes an object pointer
to accessor functions to extract internal information
or invoke member functions.
<p>
While it is possible to write accessor functions
manually, SWIG automatically creates them when it is
given structure and class definitions. For example, in the gd library, the
following structure is used to contain image information:
<blockquote><pre>
typedef struct gdImageStruct {
unsigned char ** pixels;
int sx;
int sy;
int colorsTotal;
...
} gdImage;
</pre></blockquote>
If this structure definition were placed in the SWIG
interface file, accessor functions would automatically be
created. These could then be used to extract information
about images as follows:
<blockquote><pre>
#!/usr/bin/perl
use gd;
$im = gd::gdImageCreate(400,300);
# Print out the image width
print gd::gdImage_sx_get($im), "\n";
</pre></blockquote>
Accessor functions are also created for C++ classes and Objective-C
interfaces. For example, the class definition
<blockquote><pre>
class List {
public:
List();
~List();
void insert(Object *);
Object *get(int i);
int length();
...
};
</pre></blockquote>
is translated into the following accessor functions:
<blockquote><pre>
List *new_List() {
return new List;
}
void delete_List(List *l) {
delete l;
}
void List_insert(List *l, Object *o) {
l-&gt;insert(o);
}
...
</pre></blockquote>
<h3> 3.6 Shadow Classes and Perl Objects</h3>
As an optional feature, the accessor functions created by SWIG can be
used to write Perl wrapper classes (this is enabled by running SWIG
with the <tt>-shadow</tt> option). While all the gory details can be
found in the SWIG Users Manual, the general idea is that the accessor
functions can be encapsulated in
a Perl class that mimics the behavior of the underlying object. For example,
<blockquote><pre>
package List;
@ISA = qw( example );
sub new {
my $self = shift;
my @args = @_;
$self = new_List(@args);
return undef if (!defined($self));
bless $self, "List";
my %retval;
tie %retval, "List", $self;
return bless \%retval,"List";
}
sub DESTROY {
delete_List(@_);
}
sub insert {
return $result = List_insert(@_);
}
...
</pre></blockquote>
This class provides a wrapper around the underlying object and
is said to ``shadow'' the original object.
Shadow classes allow C and C++ objects to be used from Perl
in a natural manner. For example,
<blockquote><pre>
$l = new List;
$l-&gt;insert($o);
...
$l-&gt;DESTROY();
</pre></blockquote>
For C structures, access to various attributes are provided through
tied hash tables. For the gd library, members of the
image data structure could be accessed as follows:
<blockquote><pre>
$im = gd::gdImageCreate(400,400);
$width = $im-&gt;{sx};
$height = $im-&gt;{sy};
...
</pre></blockquote>
The other significant aspect of shadow classes is that they allow Perl
to perform a limited form of automatic memory management for C/C++
objects. If an object is created from Perl using a shadow class, the
<tt>DESTROY</tt> method of that class automatically invokes the C++
destructor when the object is destroyed. As a result, C/C++ objects
wrapped by shadow classes can be managed using the same reference counting
scheme utilized by other Perl datatypes.
<h3> 3.7 Class Extension</h3>
When building object-oriented Perl interfaces, it is sometimes useful
to modify or extend objects with new capabilities. For example,
the gd library defines the following data structure for defining points:
<blockquote><pre>
typedef struct {
int x,y;
} gdPoint;
</pre></blockquote>
To make this structure more useful, we can add constructors, destructors,
and various methods to it (regardless of whether it is implemented in C or
C++). To do this, the SWIG <tt>%addmethods</tt> directive can be used as follows:
<blockquote><pre>
/* Add some methods to points */
%addmethods gdPoint {
/* Create a point or an array of points */
gdPoint(int npts = 1) {
return (gdPoint *)
malloc(sizeof(gdPoint)*npts);
}
/* Destroy a point */
~gdPoint() {
free(self);
}
/* Array indexing */
gdPoint *get(int i) {
return self+i;
}
/* A debugging function */
void output() {
printf("(%d,%d)\n",self-&gt;x,self-&gt;y);
}
};
</pre></blockquote>
Now, in the Perl interface <tt>gdPoint</tt> will appear just like a
class with constructors, destructors, and methods. For example,
<blockquote><pre>
# Create a point
$pt = new gdPoint;
$pt-&gt;{x} = 20;
$pt-&gt;{y} = 50;
$pt-&gt;output();
# Create an array of points
$pts = new gdPoint(10);
for ($i = 0; $i &lt; 10; $i++) {
$p = $pts-&gt;get($i);
$p-&gt;{x} = $i;
$p-&gt;{y} = 10*$i;
}
# Pass the points to a function
gd::gdImagePolygon($im,$pts,10,1);
...
</pre></blockquote>
The class extension mechanism is also a powerful
way to repackage existing functionality. For example, the
<tt>gdImage</tt> structure and various functions in the gd library
could be combined into a Perl class as follows:
<blockquote><pre>
%addmethods gdImage {
gdImage(int w, int h) {
return gdImageCreate(w,h);
}
~gdImage() {
gdImageDestroy(self);
}
int
colorAllocate(int r, int g, int b) {
return gdImageColorAllocate(self,r,g,b);
}
void
line(int x1,int y1,int x2,int y2,int c) {
gdImageLine(self,x1,y1,x2,y2,c);
}
...
};
</pre></blockquote>
Users can now write scripts as follows:
<blockquote><pre>
#!/usr/bin/perl
use gd;
$im = new gdImage(400,400);
$black = $im-&gt;colorAllocate(0,0,0);
$white = $im-&gt;colorAllocate(255,255,255);
$im-&gt;line(20,50,180,140,$white);
...
</pre></blockquote>
With these simple modifications, our interface is already
looking remarkably similar to that used in the GD module
on CPAN. However, more improvements will be described shortly.
<h3> 3.8 Access Control and Naming</h3>
In certain instances, it may be useful to restrict access to
certain variables and class members. Hiding objects is easy--simply
remove them from the interface file. Providing read-only access can
be accomplished using the <tt>%readonly</tt> and
<tt>%readwrite</tt> directives. For example,
<blockquote><pre>
// Create read-only variables
%readonly
int foo; // Read-only
double bar; // Read-only
%readwrite
// Create read-only class members
class List {
...
%readonly
int length; // Read-only member
%readwrite
...
}
</pre></blockquote>
When read-only mode is used, attempts to modify a value from
Perl result in a run-time error.
<p>
Another common problem is changing the name of various C declarations.
For example, a C function name may conflict with an existing Perl keyword or subroutine. To
fix this problem, the <tt>%name</tt> directive can be used. For example,
<blockquote><pre>
%name(cpack) void pack(Object *);
</pre></blockquote>
creates a new command ``cpack.'' If name conflicts occur repeatedly,
the <tt>%rename</tt> directive can be used to change all future occurrences
of a particular identifier as follows:
<blockquote><pre>
%rename pack cpack;
</pre></blockquote>
The renaming operations can also be applied to C/C++ class and structure
names as needed. For example,
<blockquote><pre>
%name(Image) class gdImage {
...
}
</pre></blockquote>
<h3> 3.9 Exception Handling </h3>
To catch errors, SWIG allows users to create user-defined exception
handlers using the <tt>%except</tt> directive. These handlers are
responsible for catching and converting C/C++ runtime errors into Perl
errors. As an example, the following error handler can be used to
catch errors in the standard C library:
<blockquote><pre>
%except(perl5) {
errno = 0;
$function
if (errno) {
die(strerror(errno));
}
}
</pre></blockquote>
When defined, the exception handling code is placed into all of the
wrapper functions. In the process, the <tt>
$function</tt> token is replaced by the actual C function call. For the
example shown, the exception handler resets the <tt>errno</tt> variable
and calls the C function. If the value of <tt>errno</tt> is modified to
a non-zero value, an error message is extracted from the C library
and reported back to Perl.
<p>
While catching
errors in the C library has been illustrated, exception handlers
can also be written to catch C++ exceptions or to use any
special purpose error handling code that might be present in an
application.
<h3> 3.10 Typemaps</h3>
Typemaps are one of SWIG's most powerful features and the primary
means of customization. Simply stated, a
typemap is a small bit of C code that can be given to SWIG to modify
the way that it processes specific datatypes.
For instance, Perl arrays can be converted into C arrays, Perl
references can be substituted for pointers, and so forth. This
section briefly introduces typemaps and their use. However, typemaps
are a complicated topic so it is impossible to cover all of the details
here and interested readers are strongly advised to consult the SWIG documentation.
<h4> 3.10.1 Example: Output Values</h4>
As a first typemap example, consider a function that returns values through its
parameters as follows:
<blockquote><pre>
void
imagesize(gdImagePtr im, int *w, int *h) {
*w = gdImageSX(im);
*h = gdImageSY(im);
}
</pre></blockquote>
As is, this function would be difficult to use because the
user must write helper functions to manufacture, dereference,
and destroy integer pointers. These functions might be used as follows:
<blockquote><pre>
$wptr = new_integer(); # Create an 'int *'
$hptr = new_integer();
imagesize($im, $wptr, $hptr);
$w = integer_value($wptr); # Dereference
$h = integer_value($hptr);
delete_integer($wptr); # Clean up
delete_integer($hptr);
</pre></blockquote>
A more elegant solution is to use the SWIG typemap library
in the interface file as follows:
<blockquote><pre>
%include typemaps.i
void imagesize(gdImagePtr im, int *OUTPUT,
int *OUTPUT);
</pre></blockquote>
Now, in the Perl script, it is possible to do this:
<blockquote><pre>
($w,$h) = imagesize($im);
</pre></blockquote>
In a similar spirit, it is also possible to use
Perl references. For example:
<blockquote><pre>
%include typemaps.i
void
imagesize(gdImagePtr im, int *REFERENCE,
int *REFERENCE);
</pre></blockquote>
Now in Perl:
<blockquote><pre>
# Return values in $w and $h
imagesize($im,\$w,\$h);
</pre></blockquote>
To implement this behavior, the file <tt>typemaps.i</tt> defines
a collection of typemap ``rules'' that are attached to specific
datatypes such as <tt>int *OUTPUT</tt> and <tt>int *REFERENCE</tt>.
The creation of these rules is now discussed.
<h4> 3.10.2 Creating New Typemaps</h4>
All wrapper functions perform a common sequence of internal ``operations.''
For example, arguments must be converted from Perl into a
C representation, a function's return value must be converted back into
Perl, argument values might be checked, and so forth. SWIG gives each of
these operations a unique name such as ``in'' for input parameter processing,
``out'' for returning values, ``check'' for checking values, and so forth.
Typemaps allow a user to re-implement these operations for specific datatypes
by supplying small fragments of C code that SWIG inserts into the resulting wrapper code.
<p>
To illustrate, consider the gd example. In the original interface file,
two functions were included to open and close files. These were required
because SWIG normally maps all pointers (including files) into blessed
references. Since a blessed reference is not the same as a Perl file handle,
it is not possible to pass Perl files to functions expecting a <tt>FILE *</tt>.
However, this is easily changed with a typemap as follows:
<blockquote><pre>
%typemap(perl5,in) FILE * {
$target = IoIFP(sv_2io($source));
}
</pre></blockquote>
This declaration tells SWIG that whenever a <tt>FILE *</tt>
appears as a function parameter, it should be converted using the
supplied C code. When generating wrappers, the typemap code is
inserted into all wrapper functions where a <tt>FILE *</tt> is involved.
In the process the
<tt>$source</tt> and <tt>$target</tt> tokens are replaced by the names of
C local variables corresponding to the Perl and C representations of
an object respectively. As a result, this typemap allows Perl files
to be used in a natural manner. For example,
<blockquote><pre>
open(OUT,"&gt;test.gif") || die "error!\n";
# Much better than before
gd::gdImageGif($im,*OUT);
</pre></blockquote>
Certain operations, such as output values, are implemented using
a combination of typemaps as follows:
<blockquote><pre>
%typemap(perl5,ignore)
int *OUTPUT(int temp) {
$target = &temp;
}
%typemap(perl5,argout) int *OUTPUT {
if (argvi &gt;= items) {
EXTEND(sp,1);
}
$target = sv_newmortal();
sv_setiv($target,(IV) *($source));
argvi++;
}
</pre></blockquote>
In this case, the ``ignore'' typemap tells SWIG that a parameter is
going to be ignored and that the Perl interpreter will not be
supplying a value. Since the underlying C function still needs a
value, the typemap sets the value of the parameter to point
to a temporary variable <tt>temp</tt>. The ``argout'' typemap is used to
return a value held in one of the function arguments. In this case,
the typemap extends the Perl stack (if needed), and creates a new
return value. The <tt>argvi</tt> variable is a SWIG-specific variable
containing the number of values returned to the Perl interpreter (so
it is incremented for each return value).
<p>
The C code supplied in each typemap is placed in a private scope that
is not visible to any other typemaps or other parts of a wrapper function.
This allows different typemaps to be used simultaneously--even if they
define variables with the same names. This also allows the same typemap
to be used more once in the same wrapper function. For example, the previous
section used the <tt>int *OUTPUT</tt> typemap twice in the same function
without any adverse side-effects.
<h4> 3.10.3 Typemap Libraries</h4>
Writing new typemaps is a somewhat magical art that requires knowledge of
Perl's internal operation, SWIG, and the underlying application.
Books such as <em>Advanced Perl Programming</em> and the man pages on
extending and embedding the Perl interpreter will prove to be quite useful.
However, since writing typemaps from scratch is difficult, SWIG provides a
way for typemaps to be placed in a library and utilized without
knowing their internal implementation details. To illustrate,
suppose that you wanted to write some generic typemaps for checking the
value of various input parameters. This could be done as follows:
<blockquote><pre>
// check.i
// typemaps for checking argument values
%typemap(perl5,check) Number POSITIVE {
if ($target &lt;= 0)
croak("Expected a positive value");
}
%typemap(perl5,check) Pointer *NONNULL {
if ($target == NULL)
croak("Received a NULL pointer!");
}
...
</pre></blockquote>
To use these typemaps, a user could include the file <tt>check.i</tt>
and use the <tt>%apply</tt> directive. The <tt>%apply</tt> directive simply
takes existing typemaps and makes them work with new datatypes. For
example:
<blockquote><pre>
%include check.i
// Force 'double px' to be positive
%apply Number Positive { double px };
// Force these pointers to be NON-NULL
%apply Pointer NONNULL { FILE *,
Vector *,
Matrix *,
gdImage * };
// Now some functions
double log(double px); // 'px' positive
double dot_product(Vector *, Vector *);
...
</pre></blockquote>
In this case, the typemaps we defined for checking
different values have been applied to a variety of
new datatypes. This has been done without having to
examine the implementation of those typemaps or having
to look at any Perl internals. Currently, SWIG includes
a number of libraries that operate in this manner.
<h3> 3.11 Other SWIG Features</h3>
SWIG has a number of other features that have not been discussed.
In addition to producing wrapper code, SWIG also produces
simple documentation files. These describe the contents of a
module. In addition, C comments can be used to provide descriptive
text in the documentation file. SWIG is also packaged with a
library of useful modules that include typemaps and interfaces to
common libraries. These libraries can simplify the construction
of scripting interfaces.
<h3> 3.12 Putting it All Together</h3>
In the first part of this section, a minimal interface to the
gd library was presented. Now, let's take a look at a more
substantial version of that interface.
<blockquote><pre>
// gd.i
%module gd
%{
#include "gd.h"
%}
// Make FILE * work
%typemap(perl5,in) FILE * {
$target = IoIFP(sv_2io($source));
}
// Grab the gd.h header file
%include "gd.h"
// Extend the interface a little bit
%addmethods gdImage {
gdImage(int w, int h) {
return gdImageCreate(w,h);
}
~gdImage() {
gdImageDestroy(self);
}
... etc ...
};
%addmethods gdPoint {
... etc ...
}
// Wrap the fonts (readonly variables)
%readonly
%include "gdfontt.h"
%include "gdfonts.h"
%include "gdfontmb.h"
%include "gdfontl.h"
%include "gdfontg.h"
%readwrite
</pre></blockquote>
Finally, here is a simple script that uses the module. Aside from
a few minor differences, the script is remarkably similar to the first
example given in the standard GD module documentation.
<blockquote><pre>
use gd;
$im = new gdImage(100,100);
$white= $im-&gt;colorAllocate(255,255,255);
$black= $im-&gt;colorAllocate(0,0,0);
$red= $im-&gt;colorAllocate(255,0,0);
$blue= $im-&gt;colorAllocate(0,0,255);
$im-&gt;transparentcolor($white);
$im-&gt;interlaced(1);
$im-&gt;rectangle(0,0,99,99,$white);
$im-&gt;arc(50,50,95,75,0,360,$blue);
$im-&gt;fill(50,50,$red);
open(IMG, "&gt;test.gif");
$im-&gt;gif(*IMG);
close(IMG);
</pre></blockquote>
<h2> 4 Interface Building Strategies</h2>
SWIG simplifies the construction of Perl extensions because it hides
Perl-specific implementation details and allows programmers to
incorporate C/C++ applications into a Perl environment using familiar
ANSI C/C++ syntax rules. In addition, SWIG interfaces are generally
specified in a less formal manner than that found in XS or component
architectures such as CORBA and COM. As a result, many users are
surprised to find out how rapidly they can create Perl interfaces
to their C/C++ applications. However, it is a misperception to think
that SWIG can magically take an arbitrary C/C++ header file and
instantly turn it into a useful Perl module. This section describes
some of the issues and solution strategies for effectively using SWIG.
<h3> 4.1 Wrapping an Existing Program </h3>
Building a Perl interface to an existing application generally involves
the following steps :
<ol>
<li> Locate header files and other sources of C declarations.
<li> Copy header files to interface files.
<li> Edit the interface file and add SWIG directives.
<li> Remove or rewrite the application's <tt>main()</tt> function if necessary.
<li> Run SWIG, compile, and link into a Perl extension module.
</ol>
While it is theoretically possible to run SWIG directly on a C header
file, this rarely results in the best scripting interface. First, a
raw header file may contain problematic declarations that SWIG doesn't
understand. Second, it is usually unnecessary to wrap every function and
variable in a large library. More often than not, there are internal
functions that make little sense to use from Perl. By copying header
files to a separate interface file, it is possible to eliminate these
functions and clean things up with a little editing [footnote: An
alternative approach to copying header files is to modify the header
files using conditional compilation to add SWIG directives or to
remove unnecessary functions]. Finally, the underlying application
may require a few slight modifications. For example, Perl supplies
its own <tt>main()</tt> function so if an application also contains <tt>
main()</tt>, it will have to be removed, rewritten, or not linked into the
extension module.
<h3> 4.2 Evolutionary Interface Building</h3>
After a Perl interface is first built, its use will
expose any problems and limitations. These problems include functions
that are awkward to use, poor integration with Perl datatypes, missing
functionality, and so forth. To fix these problems,
interface files can be enhanced with helper functions,
typemaps, exception handlers, and other declarations. Since
interfaces are easily regenerated, making such changes is a relatively
straightforward process. However, as a result, SWIG interfaces tend
to be built in an evolutionary and iterative manner rather than being
formally specified in advance.
<h3> 4.3 Traps and Pitfalls</h3>
Finally, there are a number of subtle problems that sometimes arise
when transforming a C/C++ program into a Perl extension module. One
of these problems is the issue of implicit execution order
dependencies and reentrant functions. From the Perl interpreter, users will be
able to execute functions at any time and in any order. However, in
many C programs, execution is precisely defined. For
example, a precise sequence of function calls might be performed to
properly initialize program data. Likewise, it may only be valid to
call certain functions once during a single execution. From Perl,
it is easy for a user to violate these constraints--resulting in
a potential program crash or incorrect behavior. To fix these
problems, applications can sometimes be modified by introducing additional
state variables. For example, to prevent repeated execution, a function
can be modified as follows:
<blockquote><pre>
void foo() {
static int called = 0;
if (called) return;
...
called = 1;
}
</pre></blockquote>
It is also possible to catch such behavior using exception handlers. For example,
<blockquote><pre>
%except(perl5) {
static int called = 0;
if (called)
croak("Already executed!\n");
$function
called = 1;
}
// List all non-reentrant functions
void foo();
...
// Clear the exception handler
%except(perl5);
</pre></blockquote>
Another common problem is that of improper memory management. As previously mentioned,
SWIG extensions use the same memory management techniques as C. Therefore, careless use
may result in memory leaks, dangling pointers, and so forth.
A somewhat more obscure memory related problem is caused when a C program
overwrites Perl data. This can be caused by a function such as the following:
<blockquote><pre>
void geterror(char *msg) {
strcpy(msg,strerror(errno));
}
</pre></blockquote>
This function copies a string into memory pointed to by <tt>msg</tt>. However, in the
wrapper function, the value of <tt>msg</tt> is really a pointer to data buried deep
inside a Perl scalar value. When the function overwrites the value, it corrupts
the value of the Perl scalar value and can cause the Perl interpreter to crash
with a memory addressing error or obscure run-time error. Again, this sort of problem
can usually be fixed with the use of typemaps. For example, it is possible to turn the
<tt>msg</tt> parameter into an output value as follows :
<blockquote><pre>
// Use a temporary array for the result
%typemap(perl5,ignore)
char *msg (char temp[512]) {
$target = temp;
}
// Copy the output into a new Perl scalar
%typemap(perl5,argout) char *msg {
if (argvi &gt;= items) {
EXTEND(sp,1);
}
$target = sv_newmortal();
sv_setpv($target,$source);
argvi++;
}
</pre></blockquote>
<h2> 5 Applications </h2>
SWIG is currently being used in an increasing variety of applications.
This section describes some of the ways in which has been used. A number
of advanced SWIG/Perl interfacing techniques such as typemaps and
callback functions are also described.
<h3> 5.1 Electronic CAD</h3>
SWIG plays a pivotal role in the development process of Badger, an
electronic computer-aided design system, being developed by Fusion
MicroMedia, used in the design of integrated circuits and other
electronic components. Badger is a fully object-oriented, modular,
and highly extensible system, running under various flavors of the
UNIX operating system as well as
Windows-NT.
<p>
The core components in Badger are constructed in C++ and are
delivered as a set of shared (dynamically loaded) libraries. The
libraries are <em>not</em> directly linked into an executable program.
Instead, each library comes with an extension language (EL)
interface that is generated by SWIG, allowing the library to be used
within a Perl program [footnote: For now, Perl is the only supported
extension language. Tcl and Java will be supported in the future].
The combination of a powerful EL and well-tuned, application-specific
software results in a system that is potent, flexible, and easy to
use.
<p>
For the most part, SWIG is used in a ``normal'' fashion: a description
of the classes contained within a library is presented to SWIG, and it
generates an EL interface that allows the code within that library to
be accessed from an EL. There are two interesting facets to the use
of SWIG within Badger: the use of ``smart references,'' and the use
of callbacks from C++ to the EL,
<h4> 5.1.1 Smart References</h4>
Suppose a Perl program calls a function defined by Badger (and wrapped
with SWIG) in order to create and return some object. Any Perl
variable used to refer to that object really holds a <em>handle</em> to
the object, implemented as a blessed reference containing the object's type and
its memory address. Although the implementation is a bit more
involved, the handle, in effect, acts like a pointer in C. Now,
suppose another function within Badger is called that causes the
original object to be destroyed. Severe problems will occur if the
Perl variable is supplied to another Badger function, because the
variable refers to a non-existent object. The reason for the
difficulty is that the extension language expects to have control over
the lifetime of the object, but the external system (Badger) cannot
meet this expectation.
<p>
It is possible to design Badger so that the extension language
has complete control over the lifetime of all the objects within the
system. Unfortunately, this approach results in a system that is too
closely tied to the implementation of a particular language, and adding
a new extension language to the mix is difficult. An alternate
solution that is simple to implement and is portable, is to
introduce ``smart references'' (also called proxies) into the
design [5, pg. 207]. In effect, a smart reference is an object that has the same
set of operations as a ``real'' object, but the smart reference's
implementation consists solely of a single pointer to a ``real''
object of the appropriate type.
<p>
The extension language interfaces within Badger have been crafted so
that the extension language manipulates smart references and that the lifetime
of a smart reference is completely under the control of the
extension language. Under most circumstances, the extension language
performs an operation on the smart reference, and the smart reference
then attempts to transfer the operation to the real object. If the
real object has been destroyed then the smart reference will have been
invalidated (it points to <tt>nil</tt>). In this case, the operation
is aborted and, if possible, an exception is raised in the extension
language. Badger contains the necessary machinery to invalidate any
smart references that point to an object being destroyed.
<p>
Modern C++ compilers, with their support for templates, run-time type
identification, and so forth, provide the means to automatically
construct smart reference classes. For a variety of reasons, we are
not able to always utilize modern compilers. Hence, we have created
the implementations of the smart references manually, which is a
tedious process. Fortunately, this task can be mostly automated by
creating our own code generator as part of SWIG. This is a simple
matter, as SWIG is a modular software system.
<h4> 5.1.2 Callbacks</h4>
The extension language interface produced by SWIG allows functions
defined in the external system to be called from within an extension
language. Unfortunately, the interface produced by SWIG does not
support the calling of extension language functions within C,
C++, or Objective-C. The ability to invoke functions bidirectionally
is needed by Badger, so support for callbacks from C++ to Perl has been
developed [footnote: For now, callbacks only work with Perl. Support
for callbacks with Tcl and Java will be added later]. The basic
approach is this:
<ul>
<li> Define a function.
<li> Register the function.
<li> Perform some operation that causes the registered function to be
invoked.
</ul>
To make this work, Badger provides an abstract base class in
C++ called <tt>Trigger</tt>, so called because a function associated
with objects of this class is invoked when an event of some kind
occurs. Badger also provides the machinery to associate <tt>Trigger</tt>
objects with an event name and with one or more objects internal to
the system. When an internal object ``receives'' an event, it
examines the set of registered functions looking for a match. If a
match is found then the <tt>Trigger</tt> object is invoked, and the name of the
event and the object that received the event are supplied as
arguments.
<p>
Badger provides a number of classes derived from <tt>Trigger</tt> that
specialize its behavior for certain extension languages, for C++, or
for an object request broker. For example, the <tt>Perl5Trigger</tt>
class is derived from <tt>Trigger</tt> and it specializes its base class by
storing a pointer to a Perl function reference (an <tt>SV*</tt>), and
by providing the machinery needed to invoke that Perl function.
<p>
For example, consider the following Perl fragment:
<blockquote><pre>
sub MyFcn {
my $EventName = shift;
my $Object = shift;
# ... rest of function here.
}
my $Object = BadgerFunction(....);
my $Name = "Can't find file";
Badger::RegisterByObject($Name,
$Object, \&MyFcn);
$Object-&gt;ReadFile("A bogus file name");
</pre></blockquote>
The <tt>MyFcn()</tt> Perl function is the callback (trigger) function,
and it is registered with <tt>$Object</tt> using the
event name called ``<tt>Can't find file</tt>''. Now, suppose that the
<tt>$Object-&gt;ReadFile()</tt> operation fails. Internally, Badger will
note the failure, determine the appropriate event name, attempt to
find a Trigger object associated with that event, and if found, will
``invoke the Trigger'' by calling the appropriate member function.
For the example above, this means that the <tt>MyFcn()</tt> function
will be called with <tt>$Object</tt> and ``<tt>Can't find file</tt>''
supplied as arguments. The function may require more information such
as the file name (that could not be opened), and it might find this
information by ``pulling'' data from the external library using the
functions wrapped by SWIG.
<p>
The <tt>RegisterByObject()</tt> function is responsible for creating
an object of the <tt>Perl5Trigger</tt> class, and for creating the association
between the <tt>Perl5Trigger</tt>, the event name, and the object receiving the
event. There is a bit of typemap trickery involved when intercepting
the arguments from Perl:
<blockquote><pre>
%typemap(perl5,in) SV* pFcn {
if (!SvROK($source))
croak("Expected a reference.\n");
$target = SvRV($source);
}
void
RegisterByObject(const char* pcEventName,
Ref* pRef, SV* pFcn);
</pre></blockquote>
The final portion of the system left to describe is the implementation
of the <tt>Perl5Trigger::Invoke()</tt> member function, which is
responsible for calling the Perl function from the C++ side of the
world. The implementation of this, taken nearly verbatim from the
Advanced Perl Programming book [1, pg. 353], looks like
this:
<blockquote><pre>
bool
Perl5Trigger::
Invoke(const char* pcEventName,
void* pObject,
const char* pcTypeName) {
dSP;
ENTER;
SAVETMPS;
PUSHMARK(sp);
SV* pSV = sv_newmortal();
sv_setpv(pSV, (char*) pcEventName);
XPUSHs(pSV);
pSV = sv_newmortal();
sv_setref_pv(pSV, (char*) pcTypeName,
pObject);
XPUSHs(pSV);
pSV = sv_newmortal();
sv_setpv(pSV, (char*) pcTypeName);
XPUSHs(pSV);
PUTBACK;
int n = perl_call_sv(this-&gt;pPerl5Fcn,
G_SCALAR);
SPAGAIN;
if (n == 1)
n = POPi;
PUTBACK;
FREETMPS;
LEAVE;
return n == 0 ? false : true;
}
</pre></blockquote>
<h4> 5.1.3 Benefits And Limitations</h4>
The benefits that SWIG provides to Badger are enormous:
<ul>
<li> Not counting custom code (e.g., language-specific
callbacks), an extension language interface can be developed in a
day, compared with weeks for a hand-crafted approach.
<li> SWIG supports the use of multiple extension languages with ease.
<li> The resulting solution is flexible, and the results can be
tailored to meet the needs of complex applications (e.g.,
callbacks, smart references, and so on).
</ul>
SWIG does have limitations, but so far, none of these limitations has
proven to be a real impediment. It also appears that most of these
limitations will be eradicated, once SWIG has its own extension
language interface (see Section 7).
<h3> 5.2 TCAP and SCCP from HP OpenCall</h3>
One of the well known pitfalls of systematic library testing is
the creation of a huge number of small C programs--each designed to perform
a single test. More often than not, these C programs have a lot of
common code that is copied from one test case to the
other. Testing is further complicated by the tedious process of
editing, compiling, and executing each of these programs.
<p>
To solve this problem, SWIG can be used to incorporate libraries
into Perl extension modules where test cases can be implemented as Perl scripts.
As a result, the compile-execute cycle is no longer a problem and
Perl scripts can be used to implement common parts of various test cases.
<p>
This section describes the integration of Perl with an API that is
part of a HP OpenCall telecom product developed at HP Grenoble. The
API provides access to the TCAP and SCCP layers of the SS7 protocol
and consists of about 20 function and 60 structure declarations.
Furthermore, most function parameters are pointers to deeply nested
structures such as follows:
<blockquote><pre>
typedef enum {
...
} tc_address_nature;
typedef struct {
...
tc_address_nature nature;
...
} tc_global_title;
typedef struct tc_address_struct {
...
tc_global_title gt;
...
} tc_address;
</pre></blockquote>
From a Perl users' point of view, the functionality offered
by the SWIG generated module must be not be very different from the underlying C API.
Otherwise, test writers may be confused by the Perl API and testing
will be unnecessarily complicated. Fortunately, SWIG addresses this problem
because Perl interfaces
are specified using C syntax and the resulting interface closely
resembles the original API.
<h4> 5.2.1 Creating the SWIG Interface</h3>
To wrap the C API, there were three choices: copy
and modify the header files into a SWIG interface file, feed the
header files directly to SWIG, or write an interface file that includes
some parts of the header files. The first choice
requires the duplication of C definitions--a task that is
difficult to manage as the API evolves (since it is hard to maintain
consistency between the interface file and header files).
The second choice may work if the header files are written in a very
clean way. However, it can break down if header files are too complicated.
Therefore, a mix of header files and interfaces was utilized.
<p>
As part of the interface building process, header files were to
be included directly into interface files. This is easily done
using the <tt>%include</tt> directive, but a number of problematic
nested structure declarations had to be fixed. For example,
<blockquote><pre>
struct tcStat {
...
union {
...
struct stat_p_abort {
int value;
tc_p_abort_cause p_abort;
} abort;
...
} p;
} tc_stat;
</pre></blockquote>
To make this structure more manageable in SWIG, it can be split into
smaller pieces and rewritten as follows:
<blockquote><pre>
typedef struct {
int value;
tc_p_abort_cause p_abort;
} tc_stat_abort;
struct TcStat {
...
tc_stat_abort abort;
...
};
</pre></blockquote>
Such changes have no impact on user code, but they simplify the
use of SWIG.
<p>
In addition to splitting, a number of structures in the
header files were to be hidden from the SWIG compiler. While
this could be done using a simple <tt>#ifndef SWIG</tt> in the code,
this could potentially result in a huge customer problem if they also defined
a <tt>SWIG</tt> macro in their compilation process. Therefore,
conditional compilation was implemented using some clever C
comments that were parsed by vpp (See the Text::Vpp module) during
the build of the SWIG interface. For example,
<blockquote><pre>
/*
HP reserved comment
@if not $_hp_reserved_t
*/
typedef struct {
int length;
unsigned char datas[MAX_ABORT_LEN];
} tc_u_abort;
/*
@endif
*/
</pre></blockquote>
<h4> 5.2.2 Shadow Classes</h4>
By default, SWIG converts structure definitions into accessor functions
such as
<blockquote><pre>
tc_global_title *
tc_address_gt_get(tc_address *);
tc_address_nature
tc_global_title_nature_set(
tc_global_title *t,
tc_address_nature val);
</pre></blockquote>
Unfortunately, using such functions is somewhat unfriendly
from Perl. For example, to set a single value, it would be
necessary to write the following:
<blockquote><pre>
$param = new_tc_address();
tc_global_title_nature_set(
tc_address_gt_get($param),
$value);
</pre></blockquote>
Fortunately, shadow classes solve this problem by providing object-oriented
access to the underlying C structures. As a result, it is possible
to rewrite the above Perl code as follows:
<blockquote><pre>
$parm = new tc_address;
$param-&gt;{gt}{nature} = $value;
</pre></blockquote>
Needless to say, this approach is much easier for users to grasp.
<h4> 5.2.3 Customization With Typemaps</h4>
To improve the Perl interface, a number of typemaps were defined
for various parts of the interface.
One use of typemaps was in structures such as the following:
<blockquote><pre>
typedef struct {
...
tc_u_abort abort_reason;
...
} tc_dialog_portion;
</pre></blockquote>
Since <tt>tc_u_abort</tt> is defined by the structure shown earlier,
SWIG normally tries to manipulate it through pointers. However, a typemap
can be defined to change this behavior. In particular, it was
decided that testers should be able to set and get this value using
BCD encoded strings such as follows:
<blockquote><pre>
my $dialog = new tc_dialog_portion;
$dialog-&gt;{abort_reason} = '0f456A';
# Or
print "User abort reason is \
$dialog-&gt;{abort_reason} \n";
</pre></blockquote>
To do this, a typemap for converting BCD Perl strings into an
appropriate byte sequence were developed. In addition, the typemap
performs a few sanity checks to prevent invalid values.
<blockquote><pre>
%typemap (perl5,in) tc_u_abort *
($basetype temp)
{
int i;
STRLEN len;
short tmp;
char *str;
$target = &temp;
/* convert scalar to char* */
str = SvPV($source,len);
/* check if even # of char */
if ( (len % 2) != 0 ) {
croak("Uneven # of char");
}
/* set length field */
$target-&gt;length=(len/2);
if ((len/2)&gt;(sizeof($basetype)-1))
{
croak("Too many bytes in value\n");
}
for (i=0;i&lt;$target-&gt;length;i++)
{
if (sscanf(str,"%2hx", &tmp) != 1 )
croak("sscanf failed on %s, \
is it hexa ?\n",str);
$target-&gt;datas[i] = tmp;
str+=2;
}
}
</pre></blockquote>
To return the byte buffer back to Perl as a string, a somewhat simpler typemap
is used:
<blockquote><pre>
%typemap (perl5,out) tc_u_abort *
{
int i;
$target=newSVpvf("%x",$source-&gt;datas[0]);
for (i=1; i&lt; $source-&gt;length; i++) {
sv_catpvf($target,"%x",
$source-&gt;datas[i]);
}
argvi ++;
}
</pre></blockquote>
SWIG typemaps were also used to fix a few other functions. For example,
some functions required an address parameter encoded as a two-element array.
By default, SWIG wraps this parameter as a pointer, but this
leaves the Perl writer with the painful tasks of creating and
filling a C array with sensible values using the SWIG pointer library or
helper functions. Fortunately, with typemaps, it was possible to
create and set this parameter using Perl hashes as follows:
<blockquote><pre>
# $address is an ordinary perl hash
# $address will be used as an array
$address-&gt;{pc} = 10;
$address-&gt;{ssn}= 12;
...
SCCP_oamcmd($cnxId, $time, undef, $address,
$command, $cmd_parms);
</pre></blockquote>
The typemap implementing this behavior is as follows:
<blockquote><pre>
%typemap (perl5,in) SccpOamAddress* {
HV* passedHash;
SV** valuePP;
SccpOamAddress tempAddress;
if (!SvOK($source)) {
/* we were passed undef */
tempAddress[0] = 0;
tempAddress[1] = 0;
} else {
if (!SvROK($source))
croak("Not a reference\n");
if (SvTYPE(SvRV($source)) != SVt_PVHV)
croak("Not a hash reference\n");
passedHash = (HV*) SvRV($source);
valuePP=hv_fetch(passedHash,"ssn",3,0);
if (*valuePP == NULL)
croak("Missing 'ssn' key\n");
tempAddress[1] = SvIV(*valuePP);
valuePP=hv_fetch(passedHash,"pc",2,0);
if (*valuePP == NULL)
croak("Missing 'pc' key\n");
tempAddress[0] = SvIV(*valuePP);
}
$target = &tempAddress;
}
/* SccpOamAddress is returned as
{'ssn'=>ssn_value, 'pc'=>pc_value} */
%typemap (perl5,out) SccpOamAddress* {
HV* passedHash;
SV* theSsn;
SV* thePc;
thePc = newSViv((*$source)[0]);
theSsn = newSViv((*$source)[1]);
passedHash = newHV();
hv_store(passedHash,"ssn",3,theSsn,0);
hv_store(passedHash,"pc",2,thePc,0);
$target = newRV_noinc((SV*) passedHash);
argvi ++;
}
</pre></blockquote>
<h4> 5.2.4 Statistics</h4>
Table 1 shows the amount of code associated with .i files and
header files as well as the amount of code generated by SWIG (.C and .pm
files). While it was necessary to write a few .i files, the size of
these files is small in comparsion to the generated output files.
<p>
<center>
<table border cellspacing=0 cellpadding=5>
<caption align=bottom> Table 1: TCAP and SCPP Modules</caption>
<tr>
<th> Module </th>
<th> .i files </th>
<th> .h files </th>
<th> .C files </th>
<th> .pm files </th>
</tr>
<tr>
<th> TCAP </th>
<td> 434 </td>
<td> 977 </td>
<td> 16098 </td>
<td> 3561 </td>
</tr>
<tr>
<th> SCPP </th>
<td> 364 </td>
<td> 494 </td>
<td> 13060 </td>
<td> 2246 </td>
</tr>
</table>
</center>
<h4> 5.2.5 Results </h4>
Overall, SWIG saved time when providing Perl access to the TCAP and SCCP
libraries. While it took some time and hard work to write the typemaps,
the SWIG approach has several advantages compared to XS or the
pure C approach:
<ul>
<li> The interface files are quite short so if they are well documented, a new
SWIG user should not have any major problems maintaining them.
<li> A new version of the API is wrapped with a 'make' command, so there is no need to edit
any file. In most cases the interface files can remain unmodified, provided
there are no weird constructs introduced in the new version of the API.
<li> New comments added in the header files will be automatically added in the
documentation files generated by SWIG.
<li> If necessary, new helper functions may be added in the .i files without
impacting other parts of the code or typemaps. This allows a new user to do it
without reading the whole SWIG manual.
<li> Typemaps that deal with basic types or simple structures are reusable
and can be used with other APIs.
</ul>
For those who are considering SWIG's advanced features, the learning
curve is a little steep at first, but the rewards are great because
SWIG advanced features will enable you to provide an improved
interface to the Perl user.
<h2> 6 Limitations</h2>
Currently, SWIG is being used by hundreds of users in conjunction
with a variety of applications. However, the current implementation
of SWIG has a number of limitations. Some of these limitations are
due to the fact that SWIG is not a full C/C++ parser. In particular,
the following features are not currently supported:
<ul>
<li> Variable length arguments (...)
<li> Pointers to functions.
<li> Templates.
<li> Overloaded functions and operators.
<li> C++ Namespaces.
<li> Nested class definitions.
</ul>
When these features appear in a SWIG input file, a syntax error or
warning message is generated. To eliminate these warnings,
problematic declarations can either be removed from the interface,
hidden with conditional compilation, or wrapped using helper
functions and other SWIG directives.
<p>
A closely related problem is that certain C/C++ programs are not easily
scripted. For example, programs that make extensive use of
advanced C++ features such as templates, smart pointers, and overloaded
operators can be extremely troublesome to incorporate
into Perl. This is especially the case for C++ programs that override
the standard behavior of pointers and deferencing operations---operations
that are used extensively by SWIG generated wrapper code.
<p>
In addition, SWIG does not provide quite as much flexibility as <tt>xsubpp</tt> and
other Perl specific extension building tools. In order to be general purpose,
SWIG hides many of the internal implementation details of each scripting
language. As a result, it can be difficult to accomplish certain tasks. For
example, one such situation is the handling of functions where arguments
are implicitly related to each other as follows:
<blockquote><pre>
void foo(char *str, int len) {
// str = string data
// len = length of string data
...
}
</pre></blockquote>
Ideally, it might be desirable to pass a single Perl string to such a function
and have it expanded into a data and length component. Unfortunately, SWIG
has no way to know that the arguments are related to each other in this manner.
Furthermore, the current typemap mechanism only applies to single arguments
so it can not be used to combine arguments in this manner. XS, on the other hand, is more
closely tied to the Perl interpreter and consequently provides more
power in the way that arguments can be converted and passed to C functions.
<p>
Finally, SWIG is still somewhat immature with respect to its overall
integration with Perl. For example, SWIG does not fully support
Perl's package and module naming system. In other words, SWIG can create
a module ``Foo'', but can't create a module ``Foo::Bar.'' Likewise,
SWIG does not currently utilize MakeMaker and other utilities
(although users have successfully used SWIG with such tools). In
addition, some users have reported occasional problems when SWIG
modules are used with the Perl debugger and other tools.
<h2> 7 Future Directions </h2>
Future development of SWIG is focused on three primary areas. First, improved
parsing and support for more advanced C++ are being added. These additions
include support for overloaded functions and C++ namespaces. Limited support for wrapping
C++ templates may also be added. Second, SWIG's code generation abilities
are being improved. Additions include more flexible typemaps and better
access to scripting-language specific features. Finally, an extension API
is being added to the SWIG compiler. This API will allow various
parts of the SWIG compiler such as the preprocessor, parser, and code generators
to be accessed through a scripting language interface. In fact, this
interface will even allow new parsers and code generators to be implemented
entirely in Perl.
<h2> 8 Acknowledgments</h2>
SWIG would not be possible without the feedback and contributions of its users.
While it is impossible to acknowledge everyone individually, a number of
people have been instrumental in promoting and improving SWIG's Perl support.
In particular, Gary Holt provided many of the ideas used in the shadow
class mechanism. We would also like to thank John Buckman, Scott Bolte,
and Sriram Srinivasan, for their support of SWIG. We also thank the University
of Utah and Los Alamos National Laboratory for their continued support.
<h2> 9 Availability </h2>
SWIG is freely available on CPAN at <br>
<p>
<a href="http://www.perl.com/CPAN/authors/Dave_Beazley">
<tt>www.perl.com/CPAN/authors/Dave_Beazley</tt>.</a><br>
<p>
Additional information is also available on the SWIG homepage at
<a href="http://www.swig.org"><tt>www.swig.org</tt></a>. An
active mailing list of several hundred subscribers is also available.
<h2> Bibliography </h2>
[1] Sriram Srinivasan. <em> Advanced Perl Programming</em>. O'Reilly
and Associates, 1997.
<p>
[2] Scott Bolte. SWIG. <em> The Perl Journal</em>, 2(4):26-31, Winter 1997.
<p>
[3] D.M. Beazley. SWIG and Automated C/C++ Scripting Extensions.
<em> Dr. Dobb's Journal</em>, (282):30-36, Feb 1998.
<p>
[4] D.M. Beazley, SWIG Users Manual. Technical Report UUCS-98-012,
University of Utah, 1998.
<p>
[5] E. Gamma, R. Helm, R. Johnson, and J. Vlissides, <em>Design Patterns</em>.
Addison-Wesley, 1995.
</body>
</html>