back up on

External reengineering of "Catalogo Bibliografico Trentino": lessons learned.

Marco Ronchetti, Diego Feltrin, Vincenzo D'Andrea, Giancarlo Succi
Dipartimento di Informatica e Studi Aziendali - Universita' di Trento
Laboratorio di Ingegneria Informatica, Via Zeni 8 38068 ROVERETO (TN) ITALY
{Marco.Ronchetti, Diego.Feltrin, Vincenzo.Dandrea, Giancarlo.Succi}@lii.unitn.it

Many programs are now in use since more than a decade. They control valuable information, but their user interface is still based on early technology, and their use is not considered user-friendly any more (if it ever was!). Often their code is not documented, and to put the hands in their code is a very expensive and error-prone task. It would therefore be desirable to be able to apply, possibly with little effort, some aesthetic surgery on them without risking to compromise their health. In a poster presented at the WWW'95 in Darmstadt, [ron95] we suggested that WWW can provide safe and relatively inexpensive means to build a better user interface to them.

We built an interface for such an application: an information system for all the public libraries of a small region in northern Italy, well known for its beautiful mountains and ski resorts. The CBT system (Catalogo Bibliografico Trentino) was established one decade ago on a proprietary systems to serve all libraries of the Trento Province (6000 Kmq), using the Dobis-Libis system. IBM 3270 terminals were placed in the civic libraries (even in the smallest villages) and connected through an X25 network. Libraries of the local University, and research centers also belong to this system. The CBT system is also reachable via telnet through a gateway (which converts VT100 emulation in the 3270 protocol). Of course, the (character based) user interface remains the one designed one decade ago, and typically users find the system rather annoying to use, especially through the telnet connection.

We therefore built a translator which on one side establishes (and keeps alive) a permanent connection with the CBT system through some of the available telnet ports, and on the other side responds to CGI scripts invoked through the HTTP server. The system is now in use, and received thankful attention from frustrated users of the original system.

Aim of our contribution to the workshop is to share the lessons we learned in developing our interface. We will sketch here a short summary of them.

In the rest of the paper we will briefly comment the above points.

1) It is possible to build such an interface without any help from the maintainers of the original application.

As we mentioned, one important advantage given by this approach is that there is not need to modify the original code. The existing application does not recognize the translator from a standard user. When we started the project of building the new interface to the Catalogo Bibliografico Trentino, we contacted the maintainers of the application. The idea was to gather as much information as possible on the system, so as to make our work simpler. We also thought that eventually the optimal place for the translator was to reside on the same LAN as the machine providing the original service. We got friendly but evasive answers: so after what we thought to be a reasonable number of attempts we decided to go our own way.

A limited amount of reverse engineering was therefore necessary: we reconstructed the users requirements in part from the (very terse) user manual, and mostly from an exhaustive examination the behaviour of the original application. Of course, one can imagine cases where some rare behaviours (like exception raising in particular conditions) cannot be explored, and therefore the investigation of the original application cannot be exhaustive. In such cases the translator would misbehave in those rare cases, which suggests some particular care in adopting this approach when critical or sensitive data are treated.

Besides the user requirements, the reverse engineering on the original application allows to extract the basic functionality provided. The user requirements are then used to define the interface toward the user (i.e. the HTML pages, which can be either static or generated on the fly: we found necessary to use a combination of the two) while the specification of the functionality is used to code the translator.

2) Surprisingly enough, the translation process allows to define and add new functionality not (directly) provided by the original application.

Our original motivation in building the new interface was to simplify the interaction with the telnet version, which is very painful and clumsy. First, one has to go through a rather long login process, in which a password must be provided and a terminal emulation must be chosen. Sometimes users forget the public password, often they are lost in front of the technical list of possible emulations. Second, the user does not have a clear idea of what he can do, and must interact with the system through an unfriendly set of one-key commands. Third, the enter key behaves differently than the return key (which is also cause of user confusion), and the list might go on for a while.

Having performed the analysis mentioned at the previous point, we realized that we has conquered a new degree freedom: we had a list of basic functionality, which we could now consider as LEGO&trademark; building blocks. We had the possibility to build a new interface which not only masked the painful aspects of the original interface, but could also offer new services achieved as composition of the basic functions.

To better illustrate this point, we describe a trivial example. The original CBT system divides the libraries into two classes: "regular libraries", which includes mostly the civic libraries, and "special libraries" which regards university and research center libraries. When a query is performed, the user has to choose one of the two classes: to extend the query to the other class one has to repeat the whole process. It looks to us like a badly designed interface, which we were able to fix in our interface by using the available building blocks. We offer to the user the possibility to choose "both classes": in such case we perform both queries to the system, we collect and format in a uniform way the results and offer to the user the full answer. The whole process is now transparent to the user, who does not even need to know about the existence of the two classes. In fact, we could even have decided to completely drop the distinction between classes on the user's side, since we think that such distinction is rather artificial: in the end we left it as an option, in case it made sense for some user.

In addition, it is conceivable to seemlessly extend the CBT system on a local basis. A library could decide to add information about (a subset) of its books. For instance, an image of the table of contents could be stored on a local machine. The translator could be easily modified so as to integrate the CBT system with queries to a local database, providing the available additional information. In the same way any legacy system could be integrated at low cost with new information (which can use state-of-the-art technology like for instance multimedia)

3) The approach can be easily applied to applications which were not originally aware of the presence of a network.

Since the translator is built to behave like regular user, the original application does not need to be aware of the existence of a network. Therefore, an application built to be used on a local basis can be exported to the whole world. Of course, all the burden of concurrent access must be handled by the translator, and it is not a small problem as we shall discuss in the next point.

A corollary of the above statement is that the network is not necessary if the goal is just to make a better interface: one could install an HTTP server on an isolated, local machine for the sake of enabling local users to use a new interface to the old app.

4) There is a problem in converting the stateful interaction of the original application into the stateless protocol on which HTTP is based.

While the WWW protocol is stateless, the application around which the translator is wrapped can be (and typically is) stateful. This means that the operations which are available depend on history: like in a chess game, the allowed moves depend on the current positions on the checkerboard. Even if we had a single user, the problem should be solved: the situation can of course be much worse since access by multiple clients is possible through the net. An additional complication comes from the fact that each user can backtrack using her/his local cache and the "Back" button provided by most browsers. The matter is complex, and we will only sketch some solutions here.

At least three mainstreams can be followed, which differ for the way the state is saved: in the first case it is responsibility of the original app, in the second of the translator, in the third the state is saved at the user's site.

The first approach spawns a new copy of the original app for each user: the translator must identify the user and send her/his requests to the correct copy of the app. Backtracking is a problem: it must be recognized, and errors must be sent back to the user. There can be problems with dangling sessions, when the user abandons the interactions (maybe due to network problems). A timeout can gracefully close such orphan sessions.

The second approach and the third approaches can be used if it is easy to rebuild a state, i.e. if, given a state, a path is known for reaching it using a reasonable amount of resources (e.g. the state can be reached in a short time, without incurring in additional expenses like repeating expensive operations). In such case, once the translator knows the requested state, it can drive the app to that state before applying the operation requested by the user. In the second approach the translator stores and retrieves the state associated to the user. As in the previous case it is therefore necessary to identify the user, and backtracking is again a problem. In the CBT system we used the third approach: the state is coded in an invisible part of the page sent back to the user. When the user replies, the translator can read in the reply the user's state. In such case backtracking is no problem, since each page carries consistent information about the state.

To close this short section, which would need a much deeper discussion which we plan to present elsewhere, we say that whenever possible one should use the last solution, which does not require identification and allows for backtracking. The second solution should be used when the state has such a complex representation that it is not feasible to send it back and forth through the net. Finally, the first solution should be attempted only if the original application does not allow to reach an arbitrary state using a reasonable amount of resources.

5) The resulting interface cannot of course fix the bugs of the original app.

The face lifting approach cannot do miracles: if the original application is slow or buggy, a new interface will hardly make it better. The approach based on a new interface can in those cases be considered at most as a temporary patch. If instead the original application still retains its validity, it might be worth considering an upgrade of its interface.

6) The interface is fragile with respect to modifications of the original app (i.e. cosmetic variations in the original app may trigger the need to modify the translator).

The translator follows a set of rule for understanding what is the current state of the application, to code the user's requests and to decode the app's answers. Such rules can be easily broken by modifying the way the old app behaves. As a result, upgrades of the old app which somehow modify its user interface will generally require a corresponding adaptation of the translator. For this reason it is recommended that the new interface is built in agreement with the app's maintainer: otherwise, in case of a conflict, the original maintainer could unnecessarily modify the original interface undermining the efficiency of the translator.

7) The final user might find slow the new interface, due to the different expectations when using a modern interface.

Finally, a word on the effect of the new interface. We got a number of enthusiastic and thankful e-mails from the final users. However, we noticed a subtle psychological effect: we feel that the user is less inclined to excuse the slowness of the original application than she/he is with regard to the new interface. It looks like when using an old interface the user expects the system to be slow (answers come anyhow within few seconds), and does not complain too much. The new interface is only a mask of the existing system, and cannot be faster on the same operation. Actually, the overall efficiency is better, since many operations are now compacted into new functions and do not require continuos interactions with the user: productivity is certainly increased! However, seeing a GUI interface the user would expect faster responses. Although we do not think of this as of a real problem, it is an interesting observation.

Summary

In summary, in this paper we have suggested that WWW technology can be used for constructing a wrapper around old applications with an obsolete and unfriendly user interface. In such way the browsers for WWW can be used to provide enhanced capability to software systems which were created when a graphic user interface was not common. We briefly discussed our experience, based on a real case which we implemented.

References

[ron95] M.Ronchetti, V.D'Andrea, G.Succi, D.Feltrin
Third International WWW Conference, Darmstadt (D) (1995)
Face Lift: using WWW technology for an external reingineering of old applications
back up on