Although in classes, one often works on individual projects, or in small autonomous teams on group projects, this is atypical of software development projects. Product development is an organizational activity (for better or worse). Usually a project will have several developers and a project manager who must work together to produce the product. In addition, there are external influences from marketing, other business and product groups, etc. Even in open source projects, several developers must cooperate and coordinate to develop larger systems, and there is usually someone coordinating the efforts to avoid redundent work and ensure consistency.
Although it has been going on for decades. Software development and project management is still an art and a mystery. Some 60% of projects are cancelled before completion (though it is impossible to tell how many should never have been started in the first place). This, in part, accounts for the popularity of programming methodologies such as OOP, design methodolgies such as UML, design patterns, etc., and project management tools and systems. Managers and businesses are always looking for the magic bullet to make projects work all the time.
In my humble opinion, the real things you need are: A clear idea of what you are trying to build and why, and a small team of very good developers. Unfortunatly, both of these are scarce.
Stages in project development (and their outputs).
Although there is dependency in these phases. They are often running simultaneously. Especially once project is implemented, feature requests, design changes, and bug fixes are happening all continuously.
Each stage generally includes review processes.
The design and architecture aspects of this phase are beyond the scope of this course. The issues at this stage are the design and specification of the system architecture and the interfaces between both the sub-system of the product and between the product and the outside world.
What makes this design phase particularly difficult is that
Again, the problems of this phase are beyond the scope of this course. For this reason, in Problem Set 3, we are implementing an architecture in which most of the design work has been done. There is a published specification and several reference implementations to compare against.
The rest of the course has been above OOP design and programming and how to implement a given specification. Here we want to discuss the social and organizational aspects of program development. We will only comment that there is that boundary between the design and implementation phases depends on the design moethodology, the properties of the team and manager, and the culture and policies of the larger organization. In some projects, the design phase is very detailed to the point of specifying every class and it's methods. Implementation is then a matter of coding. In other projects, the design is specified at a much higher level, and the detailed implementation left to developers. In either case, the high level modules and their interactions and interfaces should be worked out in the design phase. These can then be distributed among the development team for implementation.
Once a project reaches a certain size or includes more than one developer, a number of new issues arise.
These days any non-trivial project uses some sort of source control system to coordinate multiple developers. There are a variety of popular systems, some proprietary, some public. Some a real hassle to use, some relatively painless.
CVS is a source control system that is relatively popular in both open-source and commercial projects, probably because it is open-source, free, reliable, available on many platforms, and relatively painless to use. CVS differs from some other systems in that developers do not explicitly lock the files that they are working on (which prevents other developers from changing them). rather each developer can check out a complete copy of the source tree and modify any or all of it. (This an important feature; it is very frustrating to find that the file you need to modify to continue has been locked by someone who has left for a long weekend.).
CVS maintains a central repository of source files and keeps a full version history so that any previous state of the project can be extracted. Developers interact with the repository through a series of commands (there are both command-line and GUI interfaces to CVS, though they implement the same commands). Amount these commands are:
The javac compiler purports to be a build system, but at present I believe it is insuffucient for many projects' needs. There is a project at Apache called ANT which claims to be a build system for Java.
One popular open build system is Make, developed originally for UNIX but now widely available, albeit in somewhat incompatable versions. Major properties of make:
make -f Makefile targetwhere target is the output desired. If the rules file is indeed called Makefile, the '-f Makefile' can be omitted.
Sample make file.
# Use variable to hold compile flags JFLAGS = -classpath mypath # Variable to hold the list of class files we need to produce CLASSES = Complex.class \ Polynomial.class \ # main target, causes all files in CLASSES to be made all: $(CLASSES) # Rules to produce each Complex.class # Rules consist of target: dependencies # commands to execute # Complex.class: Complex.java javac $(JFLAGS) Complex.java # General rules to convert .class to .java %.class:%.java javac $(JFLAGS) @< # A target to remove non-source files clean: rm *.class
Another build-related issue is how to organize source and object files. For a small project, everythings can live is one directory. However, for a larger project, this becomes cumbersome. There is no general rule for organizing files, though creating a separate directory for each module is usually a good idea.
Java packagesJava intermingles class/method namespace issues with directory organization (at least for class files). A classes in a package named 'foo' will be be expected in subdirectory 'foo' relative to the classpath. (This is probably due to an ill-concevied attempt to impose a Java-centric class oriented view on users and developers, it ends up being simply annoying until eliminated with .jar files.) At any rate, this imposes some constraint on how object files are organized. It is probably a good idea to keep source files in the same hierarchy.
Periodic buildsIn a large on-going project, it is generally a good idea to build the latest version of system (as it exists in the source repository) periodically (usually nightly). This is checks that no-one has check in code that is incompatible with the rest of the system (eg changed an interface). If the nightly build fails, a search ensues for the guilty party.
DocumentationDocumentation is an integral part of the software development process. One can identify several classes of documentation depending on the intended audience and level of detail.
In general, user and installation documentation is written by professional tech-writers (if you are lucky) with input from the designers and developers, as well as marketing and sales. We will not discuss this further.
Module level documentation is critical both for successful project completion (as it specifying design and functionality) and future maintainance. It can also form the main body of User-level API documentation. It should include, at minimum, a description of each of the classes and interfaces, what they are supposed to represent and do, and descriptions of each of their methods. It should be suffient for a developer unfamiliar with the project to user the module without examining the source code.
Implementation documentation is similar to module documentation but at a lower level. The degree of implementation documentation required is more flexible. In general, classes and important methods should be documented. An overview of the implementation should also be provided covering data representation and any algorithms used. As for in-line documentation, the code plus comments should be a readable description of what the code is doing; it should tell it's story (as well as compiling to a working program). If the code is straightforward and clear, minimal comments are required. If the implementation is in any way tricky or obscure (eg for optimization), provide an explanation of what is going on.
Module 1, 2, and 3 don't depend on (call into) any other modules. Module 4 calls modules 1 and 2. Module 5 calls modules 3 and 4.In this case, Modules 1,2,and 3 can be tested independently. Then module 4 tested using modules 1 and 2. Any errors found in testing this combination should be confined to module 4 since then others have been already verified (knock on wood). Similarly 5 can be tested using verified versions of 1,2,3 and 4.
If the dependency graph includes cycles, testing is more complex. Several modules may have to be tested as a unit.