A few weeks ago, Matt Gibson posted on the asterisk-users discussion list that he was looking for a GraphViz script he had heard about a few years ago that could generate graphs of Asterisk's extensions.conf file that defines the dialplan. At the time, I had been working on a recently open-sourced application, EgoNet, for a researcher I work with. This application taught me much about social networks, particularly his specialty of egocentric networks, and it also helped me learn a ton about the Java Universal Network/Graph Framework (or JUNG). I immediately thought about combining my experience with JUNG and Asterisk, and wrote a rudimentary (read as: extremely ugly) parser for extensions.conf and generated a crude graph of the sample dialplan.

Read more...

Amazon offers a few massivly scalable web services like Elastic Compute Cloud (EC2) and Simple Storage System (S3). EC2 delivers virtual Linux boxes and S3 offers fail-safe disk space. Those services can be used as building blocks for custom applications. They are well thought out and allow you to start small and grow fast while concentrating on your own business (application) and leaving the operational details to a third party.

A rather new service and still in limited beta is Amazon SimpleDB. It is "a web service for running queries on structured data in real time." While S3 stores unstructed data like files, SimpleDB stores items with attributes organized in domains. The concept is a similar to other databases where an item could roughly be described as a row, an attribute as a column and a domain as a table. There are no connections between the domains (i.e. no foreign keys or referential integrity) but you can run arbitrary queries on the attributes of the items in a domain.
Like all Amazon webservices SimpleDB is accessible through HTTP using either a SOAP or a REST-style API. You can GET, PUT or DELETE items in a domain along with their attributes and you can QUERY a domain with a simple set of operators: =, !=, <, > <=, >=, STARTS-WITH, AND, OR, NOT, INTERSECTION AND UNION. A QUERY returns a list of item identifiers and is quite fast. If you also need the attributes of these items however you must execute an additional GET request for each item that has been returned.

What does that mean in practice?

Say you store a set of blog entries in SimpleDB and want to show a list of the ten least recently added entries with title, date and author. You would issues a QUERY request that returns the identifiers of the ten entries and then issue ten GET requests to retrieve the title, date and author attributes for each of them. That makes a total of 11 requests that have to be executed. In general you need n+1 requests for a query that returns n items.

Amazon explains that you can run these queries in parallel to improve the overall response time for a query but that means you are running 50 or 100 parallel requests for just 10 visitors looking at your list. Sure, you can also cache them or do some other fancy tricks but the problem remains: Any query to SimpleDB requires n+1 HTTP requests. The bigger your result sets the worse your performance. Interestingly enough the n+1 problem is well known (and dreaded) from O/R-mapping where it can hit you if relatated objects are loaded in an improper way. The difference is however that with SimpleDB you can't escape it and that an HTTP request to SimpleDB is more expensive than an additional SELECT to a conventional database.

So what's the point here?

In my opinion SimpleDB in its current shape can only be used for very few use cases if at least one of the following conditions is met:

• Your application only uses PUT, GET and DELETE requests
• Your application only uses queries that return very few items (10 or less)
• Your application only uses queries very infrequently, for example if serving all data from a distributed cache and using SimpleDB only as a backup store for structured data

Maybe that's one of the reasons for Amazons to partner with Sun and provide support for MySQL on EC2.

A recent post to the mailing list asked about how to stop and start a FastAGI server and how folks host Fast AGI servers. I shared some code I wrote for a small MBean that starts and stops a DefaultAgiServer from Asterisk-Java -- see the last FAQ question "Can I have an example of using an AGI server in a container like JBoss?". The MBean uses the JBoss-specific @Management annotation.

I work on a home-grown call center management application (EJBs, JSP webpages, and two standalone Java Swing applications, all talking through JBoss Application Server), and we've already been able to add a couple features with Asterisk-Java that add value and utility to our application. I didn't want to introduce another service to our infrastructure and I also didn't want to write an entirely new Agi server, as Stefan had done a great job on that part. Thus, the above MBean was born.

Comments and suggestions and other examples of how you run your Agi scripts are welcome and encouraged on our mailing lists.

Atlassian has improved the charting features in their latest release of Fisheye. So I had a look at the Asterisk-Java code base to see how it's major packages compare in size. This is what i got:

Let's have a look at the individual packages:

The Manager API

Support for the Manager API (manager) is by far the biggest package in Asterisk-Java. It provides access to a variety of Asterisk features from call control to monitoring and call center management. The size of this package and its steady groth ressemble the groth of features exposed by Asterisk.

The Asterisk Gateway Interface

FastAGI is an easy way to implement IVR applications in Java. The feature set is quite stable over time so we don't see significant changes in the code base. You'll even notice that support for AsyncAGI (i.e. running AGI scripts over the Manager API) was a quick win as it didn't have a significant impact on the overall size of the package.

Parsing Configuration Files

The config package is quite new and supports parsing Asterisk's configuration files. All config files use the same syntax and group up configuration items into contexts. It supports inheritance of contexts, includes and forms the basis for further development like configuration editors or dialplan visualization (stay tuned for more information from Martin on this topic).
Originally developed to parse voicemail meta files the config package is certainly a candidate for future growth.

The Live API

The live API is our own invention and takes integration a step further. Instead of only providing access to the basic features exposed by Asterisk the live API offers real objects with state and behavior that model core concepts found in Asterisk. There are channel objects with state like the current caller id, the progress or hangup reason and methods to redirect channels, start and stop monitoring and much more.
The live API speeds up developers by hiding the sometimes tedious details of the low level Asterisk APIs and has reached a significant size. Next to the config package the live API is under active development and will certainly grow in the future.

In the past days I've notices two Open Source projects that changed their license from a commercial friendly license (BSD/MIT, LGPL) to GPL. The motivation of Java Service Wrapper and Ext JS seems to be to offer commercial licenses for closed-source projects that cannot use GPLed libraries.

While I understand that these projects need resources to enhance their products this step will have an impact on the trust companies will have in Open Source projects. It seems both projects have used a commercial friendly license to attract people. Now that they have an established user base they try to monetize this asset. It remains to be seen if this will work as various groups are already discussing the options to fork these projects, nevertheless the overall damange remains. "A foundation you can build on" - the claim of Ext JS - leaves a bad taste. Not only that companies developing closed-source products will have to pay now (or switch or stick with the old soon unsupported version), Open Source projects with a more commercial friendly license will also have to look for alternatives.

Another interesting aspect is the current use of the GPL: Once designed to grant a maximum of freedom to the users of software it has become a means to restrict usage. Many projects that use the GPL choose to dual license their software and sell commercial licenses to those who can't use GPLed libraries in their products.

Mauro has published an interesting article about Asterisk and SugarCRM integration. A swing popup notifies its user on incoming calls and provides additional information about the customer that is retrieved from SugarCRM.

The sample application serves as a good example on how to use Asterisk's Manager API with Asterisk-Java.

References

• Mauro: SugarCRM and Asterisk integration in Java

The enterprise edition of the popular Java-based XMPP server Openfire is becoming Open Source. This also includes the Flex-based IM client Sparkweb.

As Matt points out the the clustering functionality in the enterprise edition will not be made Open Source: "Part of the reason for this is that it uses a third-party commercial library for clustering."
Nevertheless, due to the moduluar design of Openfire, the use of Coherence can quite easily be substituted by a free alternative like Terracotta.

This is really great news. Thanks to Jive for their commitment to the community.

Update 2008-04-07

Dombiak Gaston has followed up with a roadmap describing the two phases of the transition. The first phase with the majority of features is scheduled be finished by April, 27th.

I've just finished adding support for Asynchronous AGI to Asterisk-Java. AsyncAGI allows you to run AGI scripts through a Manager API connection.

The way AsyncAGI is supported by Asterisk-Java hides the differences in the underlying communication from the users of our library. Your AGI scripts developed for FastAGI will run with AsyncAGI without a change.

To make use of AsyncAGI add the following extension to you dialplan:

exten => 1234,1,Agi(agi:async)

Create a simple AGI script, pass it to AsyncAgiServer and register the AsyncAgiServer as a listener to a ManagerConnection:

public class SampleScript extends BaseAgiScript
{
    public void service(AgiRequest request, AgiChannel channel) throws AgiException
    {
        channel.streamFile("tt-monkeys");
    }

    public static void main(String[] args) throws Exception
    {
        ManagerConnection connection;
        AsyncAgiServer agiServer;

        connection = new DefaultManagerConnection("localhost", "manager", "pa55w0rd");
        agiServer = new AsyncAgiServer(new SampleScript());
        connection.addEventListener(agiServer);
        connection.login();

        while (true)
        {
            Thread.sleep(1000L);
        }
    }
}

To run the sample you need the latest snapshot of Asterisk-Java 1.0.0 (at least 20080404.222056-117) and Asterisk 1.6.0.

References

• Moy: Asterisk Asynchronous AGI

Thanks to Martin we now have a FAQ section available as part of our documentation.

The FAQ covers the questions we encounter on our mailing lists and that we think will be interesting to a more general audience. Please check them before asking questions to make sure you don't ask questions again that have already been answered before.

If you notice anything we've missed feel free to post a comment or join our mailing list and propose your addition.

References

• Asterisk-Java Frequently Asked Questions

Just received a great link from Martin Smith on the Asterisk-Java dev mailing list: Help Vampires.
The page includes everything you need to know:

• How to identify them
• What to do if you are a Help Vampire
• How you can reform them

A lot of help vampires originate from countries that provide cheap outsourcing capabilities for companies in Europe and the US. Interestingly enough these vampires usually receive support from employees and contractors working for (and paid by) just the companies that enjoy the cheap prices of offshore development.

References

• Help Vampires: A Spotter's Guide

WHOIS is a simple TCP-based protocol that is widely used to provide human-readable information about domain names, registered networks, NIC handles and more. It dates from the old-ages of the Internet and is described in RFC 3912.

Usually you will use WHOIS to lookup the technical or administrative contact of a domain or network in case of technical (like DNS problems, routing trouble) or legal issues. Today I had a look at a .com domain and what I received was something like this:

$ whois ibm.com
Registrant:
International Business Machines Corporation
   New Orchard Road
   Armonk, NY 10504
   US

   Domain Name: IBM.COM

   ------------------------------------------------------------------------
   Promote your business to millions of viewers for only $1 a month
   Learn how you can get an Enhanced Business Listing here for your domain name.
   Learn more at http://www.NetworkSolutions.com/
   ------------------------------------------------------------------------

   Administrative Contact:
      DNS Admin, IBM
      IBM Corporation
      New Orchard Road
      Armonk, NY 10504
      US
      +1.9147654227 fax: +1.9147654370

Network Solutions has long been known to seek additional revenue from their NIC services by adding "creative" features, but selling ads in WHOIS (obviously available since the end of last year) just sounds crazy.

References

• DomainNameNews: Network Solutions Enhanced Whois Listings Provide Advertising to 1 Million per Day

Today I had a look at the different options to compress files (in this case for backup purposes) on a Ubuntu system. The most common tools to compress files are gzip and bzip2. They have both been around for a long time, are available on most systems by default and are nicely integrated with other utilities like GNU tar (using its -z and -j options).
7-zip and the algorithm it uses (LZMA) is not that common on UNIX-like operating systems. It is well-known as a free alternative for WinZip on Windows systems and was started back in 1998. For Ubuntu p7zip – a port of 7-zip to POSIX – is available in universe (sudo apt-get install p7zip).

My test file was a MySQL dump with a size of 163 MB that contains mostly text. I was interested in the compressed file size and in the time it takes to compress and uncompress the file.

Here are the results:

Compressor	Size	Ratio	Compression	Decompression
gzip	89 MB	54 %	0m 13s	0m 05s
bzip2	81 MB	49 %	1m 30s	0m 20s
7-zip	61 MB	37 %	1m 48s	0m 11s

For the test I ran all tools with their default settings, i.e. without providing any special options.

Gzip is still a great tool and provides good compression without consuming a lot of computation power. Bzip2 is much slower and only provides slightly better compression. 7-zip consumes a bit more cycles than bzip2 but results in far smaller compressed files. Speed for decompression is even better for 7-zip than for bzip2.

So if time is important (think of on-the-fly compression) gzip is the tool of choice. If you don't care too much about processing speed and need very good compression have a look at 7-zip. The only advantage bzip2 has over 7-zip is that bzip2 is part of most default installations and is more common. Let's hope this will change in the future, especially integration with GNU tar would be great.

References

• gzip Homepage
• bzip2 Homepage
• 7-zip Homepage
• p7zip for POSIX systems

Andreas Neugebauer's submission to the snom XML Contest was a conferencing application for Asterisk:

XMLConference uses the Asterisk Manager interface, and shows call-related information on the snom phone display and allows you to interact with these calls.

XMLConference uses Asterisk-Java's implementation of the Manager API to control Asterisk MeetMe rooms and make them available to the snom XML browser. It runs in a standard Java web container like Apache Tomcat.

References

• XMLConference inlcudes source code and binaries
• The snom XML Contest

beanizer.org has published an interesting article on how to build an AGI server with Asterisk-Java to run AGI scripts written in BeanShell.

They provide a dispatcher AgiScript that delegates to a BeanShell script and provides some additional convenience functions to make the custom scripts easy to implement.

The advantages of using scripting languages on the JVM along with Asterisk-Java are compelling:

The approach is quite flexible, our script engine doesn't need to be on the same computer the pbx is on, and we can add/modify our scripts on the fly without need for compilation or engine restart.

You might also be interested in our recent posting on writing AGI scripts in Groovy that describes a similar approach with a focus on Groovy.

References

• A simple AGI scripting engine with Asterisk-Java
• BeanShell - Lightweight Scripting for Java
• AGI scripts in Groovy

While skipping through our referrer list I stumbled upon this comment from a French user:

Ahhhh !!!
Asterisk-Java, quelle belle API. C'est la plus complète parmi celles que l'on puisse trouver pour interroger le manager ou développer des "scripts" AGI.
Coder avec cette API est vraiment d'une simplicité... :-) Bon, c'est Java... On regrettera juste qu'il n'y ai rien vraiment à la hauteur pour d'autres langages.

It seems like even developers not devoted to Java are in love with our library.
Interestingly enough there a quite few developers using other languages than Java who are using our javadocs as a general purpose reference to the Manager and FastAGI API.

Thanks for your kind works, it helps us keep up further enhancing Asterisk-Java.

Agile software development has been en vogue in this decade. It started with the popularity of Extreme Programming (XP) and Kent Beck's series of books on the topic.

One of the values stated in the Agile Manifesto is "Responding to change over following a plan" and is accompanied by the principle

Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

While I generally consider most agile practices as common sense, some of they may also cause trouble.

Being agile encourages customers to feed the development team with ideas on the spur of the moment while non-agile processes enforce proper planning. As a result of the slower time to market more formal processes prevent short-run ideas from being implemented and consuming resources that were better spent working on future-proof core requirements. Besides this those short-run requirements clutter the system if not properly removed once they are no longer needed.

A nice solution is to use differnt processes for differnt systems. Agile processes are best deployed for systems with a short lifetime and requirements that are not fully understood upfront. Examples for this kind of systems are public facing web applications like web stores. More formal processes are well suited for core systems with a long lifetime where maintenance cost is an important factor and stability is key.

This separation allows new ideas to be tested in an agile environment and move them to core systems once they become mature and turn out to be important features in the long run. When they will finally reach the core system the requirements are well understood and can easily be specified in a formal way - so the formal process is not a burden but a line of defense for feature creep.

Which systems belong to which category depends on the type of your business. If you are a bank your accounting systems will probably fit the "core" category as will the systems needed for compliance and trading. The public facing systems and systems mainly used for sales could benefit from an agile process.
The following questions may help you when you classify your own systems:

• How long do you expect the system to live?
• How stable are the requirements for the system?
• Will your system undergo frequent changes?
• How important is the system for your company in the long run?

If you are planning new systems you can also use this classification to define your system boundaries.

Since a few days the first beta of Asterisk 1.6.0 is available for download.

So far, Asterisk-Java seems to work fine with the new release.
If you encounter any problems feel free to post a comment, join our mailing list or post a bug report.

Update 2008-01-29:

...and they followed up with 1.6.0-beta2.

For the 1.4 series they start providing release candidates, an idea that was discussed on the -dev list before and aims at providing better qualitiy for the final production releases:

The release candidate for 1.4.18 is only available via svn. It is available for anyone that would like to help test 1.4.18 over the next couple of days before it gets officially released.

Grab it from http://svn.digium.com/svn/asterisk/tags/1.4.18.

Today we had an interesting bug in a small web application developed for a customer in the financial industry. The application is based on the Spring Framework, secured by Acegi Security and makes heavy use of AJAX (powered by DWR).

Everything went fine while testing with different browsers from Firefox to Safari and Internet Explorer in different versions. Finally we have started testing in the target environment - well locked down and without support for ActiveX controls from untrusted sites. IE6 needs ActiveX for its implementation of XMLHttpRequest (XHR) - the heart of AJAX. If XHR is not available DWR automatically switches to using IFrames to emulate XHR. This usually works well and has already been used in the previous version. Nevertheless the application just didn't work: Every remote call failed with a not so user friendly error message: "The specified call count is not a number: null".

Remote debugging of Tomcat showed that DWR is trying the read the request data using req.getInputStream() to parse it. When using IFrames reading from the request's input stream fails immediately and returns null, when using XHR it works fine. The main difference is the content type of the requests "application/x-www-form-urlencoded" for IFrames and "text/plain" for XHR. As IFrames do work without Acegi but fail with the Acegi filters in place I guess Acegi does mess with the requests when it wraps them in its SavedRequestAwareWrapper.

I didn't have the time to further track it down, but I created a small workaround that falls back to using req.getParameter() if reading the stream fails.

The following snippet shows the modification made to DWR's ParseUtil.java:

in = new BufferedReader(new InputStreamReader(req.getInputStream()));

while (true)
{
    String line = in.readLine();

    if (line == null)
    {
        if (paramMap.isEmpty())
        {
            Enumeration nameEnum = req.getParameterNames();
            while(nameEnum.hasMoreElements())
            {           
                String name = (String) nameEnum.nextElement();
                paramMap.put(name, req.getParameter(name));
            }           
        }
        break;      
    }
...

And who is to blame? Well, as with many interesting problems that's hard to decide. It's just a combination of multiple pieces of software mixed with environment constraints. I guess it's something that just happens and reminds us that testing is not useless.

Update 2008-02-22

Joe has just released DWR 2.0.3 that includes a fix for this issue.

In preparation for my next test I came across a very fascinating mathematical problem.

"Prove that there always two subtended points on the Earth's equator where you are able to measure the exact same temperature." (Adapted from the Book: Repetiorium der Analysis Teil 1 (en. Revision Course for Analysis Part 1) from Timmann).

I have tried to visualize the proposition. The two red dots represent two subtended points on the equator. Remember: We want to show that there are always two subtended points (the red ones) on the Earth's equator where you are able to measure the exact same temperature.



If you believe it or not: That can be proven by combining simple mathematical concepts.

Let the points of the equator be the interval , a point on the equator and a function which returns the temperature at the given point. is a continuous function, obviously. returns the difference between two subtended points on the equator. Like the temperature function this function is continuous too. Because of . If then the two subtended points on the equator must have the same temperature. If then have different signs and therefore because of the "intermediate value theorem" there exists a and this means that which was we wanted to show.

When I first came across this problem I couldn't believe what one can prove with math. I hope you enjoyed my first post. More will come.

BEA is well known for its J2EE application server Weblogic and recently extended its product line to a SOA stack branded Liquid. Oracle has its own J2EE compontents mainly derived from Orion Server and branded as OC4J. They also have a SOA stack called Fusion.

So you might ask what the future of these products will be.
Oracle already acquired three different business application suites in the past: J. D. Edwards, PeopleSoft and Siebel. Does it make sense to develop three different product lines of business application suites and at least two different product lines of Java EE and SOA middleware? Probably not.

This uncertain future may make companies reevaluate their current technology stack for middleware applications. They will notice that there is an alternative beyond the IBM Websphere dinosaur and JBoss which is now RedHat. The alternative is Apache Tomcat and Spring Framework mixed with some Terracotta if you need distributed shared data.

Spring basically allows you to assemble your own application server. Transaction management, security, remoting, O/R mapping, clustering – just add what you need when you need it. Basically there is not much left where Spring does not offer a proven solution. On the other hand the features where commercial J2EE servers have been strong in the past (compared to other stacks) are becoming less important:

• EJB remoting will be replaced by web services
• entity bean style O/R mapping has already been replaced by Hibernate (where EJB3 tries to catch on)
• load distribution and fail-over is easy in a world of HTTP or JMS based web services
• clustering of data is less important in the stateless world of services

In the end the uncertainty caused by Oracle's acquisition of BEA could very well further increase the growth of Spring and the open source components it makes so easy to combine.