Software Development is a tricky business. The standard approach is to treat software development like other engineering projects – with the expectation that engineering level predictive standards are applicable. However, software is different – much different. The differences are driven by:

· The environment of the software development project is ambiguous

· Project requirements typically change sixty (60) percent once the project vision has been created[1].

· Technical complexity leads to greater variance – projects often have to be pioneers by implementing a technical solution for the first time.

· Software development is a people intensive process – governed by random and unpredictable events and human behavior

· Ambiguity of requirements translates to increased project risk. To operate effectively within the environment of “squishy” requirements, iterations are often performed in an attempt to mitigate risk, do what can be done, and get early feedback from the customer. Requirements generate requirements – thus early iterations will likely generate additional project requirements as the customer continues to finalize the definition of what they want.

The default assumption is that a stepwise sequenced set of events occurring in a waterfall fashion will lead to project success for any given software project. Business owners, product salesman, and project managers with experience in other disciplines that lend themselves to well known, stable, and predictable environments will inevitably want to apply the techniques that are in their comfort zone when working on a software project. The methodology applied to other project domains outside of software is typically a stepwise sequenced (waterfall) approach. The application of a waterfall methodology often (due the conditions above) leaves the software project manager, project owner, and development team angry, frustrated, and often delivering late or not at all. Over thirty (30) percent of software projects are cancelled before delivery. Eighty six percent of software projects fail to deliver, deliver late, or exceed their projected budget. Sixty five (65) percent of “very large” information technology (IT) projects are cancelled prior to delivery[2].

What’s it all about

One can dramatically improve these dismal statistics by selecting a methodology that is compatible with their project. Even then, further efficiencies can be gained by further tailoring the selected methodology to the specifics of the project. This white paper describes a system and method for selecting a compatible methodology for your project. This document is a white paper version of a patent titled “System and Method for Software Methodology Evaluation and Selection”. We begin by defining something called a Project Context. Taking a Copernican view, the project context is placed in the center of the universe for all relevant software project discussions. An OO approach to describing a project is taken. A project context is made up of multiple components including People, Process, and Technology (Figure 1). Each component has multiple attributes. Component attribute values can be assigned by the key stakeholders, project manager, team members, or via an outside assessment service.

Creating and defining a methodology model was the second critical element of the underlying patent. A methodology model not only has a data element for each attribute of a project context, but also defines shadow minimum and maximum compatibility range data elements for each project context attribute. A methodology model may also have some elements unique to itself, but is guaranteed to have a common set of attributes and the following methodology model items for each project context attribute: methodology model attribute; minimum compatibility range; maximum compatibility range. Establishing minimum and maximum values within the methodology model allows concepts like simulation, statistics, and statistical forecasting to be useful. The concept of using software project context simulation along with the techniques of mathematical statistics, regression analysis, and statistical forecasting was the third critical element of the underlying patent.

Now that a project context has been defined, rule evaluation can be performed on the attribute values. Rules and rule-sets can be defined for the various component and project attributes. For example:

· Project size > 10

· Number of tiers > 3

· Product build duration > 2

The concept of project rules can be taken further. Now that attributes and rules exist, a scoring mechanism can be created. The proposition that a scoring interface exists for each methodology was the fourth critical element of the patent. Each methodology model file can be scored against the project context. Rules capture a relationship between one or more project or methodology data elements and a given value(s). A rule-set contains two or more rules. Rules and rule-sets are given nicknames. Rules can be written against data in the project context, and the methodology model (attribute, min value, max value).

Example rules utilizing a methodology model are:

· Project size > methodology model project size compatible minimum

· Number of tiers < methodology model number of tiers compatible maximum

· Number of time zones > methodology model number of compatible time zones.

Later, we shall see that rules can also be written against something called the “methodology compatibility matrix”, but we reserve that for the advanced topics section.

The concept and creation of a methodology compatibility matrix was the fifth critical element of the patent. To determine the most appropriate methodology, we score the project against methodology after methodology (model after model) to find the best (most compatible) score. If the project attribute value is within the minimum to maximum attribute values (compatibility range) specified for that methodology, points are awarded. If the project attribute value is outside the compatibility range, points are not awarded and/or are deducted. The methodology file with the greatest number of points wins – and is most compatible for that given project.

Extending rules into the realm of patterns and defining the existence of a methodology pattern framework for a software development project context was the sixth critical element of the underlying patent. Working from the rich set of data provided by the project context attributes, methodology model data elements, and compatibility matrix cell values, the rule and rule-sets that provide predictive behavior are actually methodology patterns. In addition to methodology patterns, because two of the primary components of the project context are people and process, patterns for team composition and methodology selection also emerge. A separate white paper, title “Software Development Patterns” is being published as a complementary white paper. “Software Development Patterns” shows how patterns are not just for J2EE or the technology world but can be equally applicable and useful to the business world. The specification and definition of a re-usable rule and pattern-mining framework (useful for any subject matter) is the seventh critical element of the underlying patent. The framework consists of defining a context object for the problem at hand, modeling the context, identifying rules, defining a compatibility matrix (if exists), and mining those rules for subsequent Patterns. Context analysis driven patterns on methodology, team composition, leadership, organizational behavior, human behavior, and marketing are all possible – the area of business is particularly ripe for context definition, modeling, rule definition, and pattern mining.

Applying the concepts, value to the reader

The concepts in this white paper can provide effective guidance, information, and solutions when used in the following situations:

· Software project bidding time. Before bidding on a fixed price project, model your project and assign attribute values to your project context instance. Score your proposed project model to determine the “best fit” methodology and create a more accurate and profitable bid. Simulate your project by assigning your context different attribute values and re-scoring to determine key people, process, or technology choice transition points that drive the methodology selection or subsequent list of “fits” / “misfits” best practice recommendations.

· Providing a Methodology assessment service. The service consists of one or more resources interviewing the project team members, stakeholders, and project review board members and assigning the attribute values to the project context being assessed. The skilled methodology assessment team assigns values to the project context data model being studied. From there, a manual or automated scoring process takes place to determine if the project is using a compatible methodology. A list of best practice recommendations (fit / misfit output from the methodology selection process) can then be presented to the project owners, participants, and project Steering Committee. The fit / misfit data captures all “fit” and “misfit” data during the scoring process of all methodologies. Additionally, a set of fit / misfit data is presented between the chosen best methodology and the project context being studies. Even though a best or most compatible methodology was chosen, some attribute values may still have anomalies. Knowing those anomalies in advance would be extremely valuable to the project team to further tweak the methodology and put in place risk mitigation plans.

· Assessing a current project based on:

o Best fit and/or what if analysis. The project manager can ask “what if” questions (simulations) to determine if a new attribute value will change the recommended methodology or generate new incompatibility warnings. For example, a project owner and manager have adopted extreme programming on a project with current funding of $2.8 million per year. The CIO calls the project owner and informs her that the project funding is increasing 60%, to $4.15 million. With an assumed project growth of 10 people, the project team could reassess their new Project Context to see if there is a different recommended methodology and if any new or changed fit / misfit best practice patterns / anti-patterns appear.

o The rules and rule-sets within the solution represent a built in library of software development and methodology best practices. During the scoring process, a list of attribute fits / misfits by methodology can be compiled for output. Thus, even for the recommended methodology, a list of special items to pay attention to (warnings) is available.

· Building successful software project teams. Software is a people intensive process. By examining the attributes defined in the People component of the Project Context, teams can be built that have built in risk mitigation strategies to many of the known causes of software project failures. By creating a focus on the People involved and aligning the software project with a compatible methodology, the chances for project success jump dramatically via built in risk mitigation, using a compatible methodology, and having the fit / misfits (patterns, anti-patterns) of the selected methodology (and others) explicitly identified to enable risk mitigation and team compensation strategies from day one.

While many of the concepts within this white paper are not limited to software development (in particular the re-usable pattern mining framework), we have chosen to specifically focus on that domain within the document.

Software Methodology

A software methodology is a social construction that includes the roles, skills, learning, activities, techniques, deliverables, standards, habits, and culture of an organization as it develops software. Software methodologies can fall across a range from lightweight to heavyweight. Software methodologies include, but are not limited to, SunTone Architecture Methodology, Unified Process, Rational Unified Process, Enterprise Unified Process, RUP-Lite, eXtreme Programming, Waterfall, Feature Driven Development ( FDD ) Process, DSDM, Personal Software Process, Team Software Process, Object Oriented Software Process, OPEN, and SCRUM.

One software methodology does not fit all software development circumstances just as one glove does not fit all hands. For software developers, process/project managers, and business stakeholders, it is desirable to know how to choose the best-fit methodology for a project. By tailoring and selecting a best-fit methodology, one can enhance the productivity of software teams, as well as improve upon the dismal industry statistics regarding software project failures[3].

Context, Context, who has the context

Context. It seems almost everything in the Java world has a context these days. Servlet, EJB, JNDI, Applet, SOAP, Graphics, Security, Page, Initial, and Event – they all have Context objects defined for them somewhere in the J2EE platform or Apache add-on services. So why not Project?

What is a Context?

Freedictionary.com defines “context” as

· The part or parts of something written or printed, as of Scripture, which precede or follow a text or quoted sentence, or are so intimately associated with it as to throw light upon its meaning.

That last part is interesting – “throw light” – keep that in mind as we discuss context simulation and regression analysis.

Die.net has the following definition:

· That which surrounds, and gives meaning to, something else.

“Surrounds, and gives meaning to” is interesting. Keep that in mind as we discuss an object-oriented approach to context definition. Surrounding almost sounds like a wrapper object – composition certainly comes to mind.

Earchitecture.org has the following definition:

· That which surrounds, and gives meaning to, something else

· Policy & structure

· Provides a framework for design and implementation

· Provides mechanism for understanding the impact of change

The use of the word “Policy” is interesting. Keep that in mind as we discuss the application of rules, rule management, and rule processing to project context. ATG Dynamo has a rules based product “policy server”.

Mechanism for understanding the impact of change is also interesting. Keep that in mind as we discuss context modeling and simulation, what if analysis, and applying the techniques of statistical forecasting to context.

Project Context

What is a Project Context?

For project context, we are taking an OO modeling approach to the software project domain. Just as a car may have components such as door, engine, and radio, a software project is made of components. Three significant components are:

· People

· Process

· Technology

Each component has a set of attributes. How does one find, discover, or locate these attributes for components of a software project context? In a nutshell, they are the attributes of a software development project that influence a project or whose values offer particularly strong explanatory behavior on the outcome of the software development project. If performing statistical analysis on a prior project, these are the set of attributes that someone skilled in the art of regression analysis would be interested.

Example attributes for People component are shown in Table 1:

Attribute Name	Description	Measurement
Accessibility of requirements providers	Overall delay and strength of communication index between the development team and the customer representatives providing the requirements	Seven point scale
Size of project	Number of people on the project	Numeric
Senior Developer Ratio	The number of Senior Developers compared to the total number of developers on the project	Seven point scale ( 0 – 5%, 6% - 11%, 12% - 16%, 16-20%, 21 – 25%, 25 – 29%, > 30% )
Communication index	A derivative attribute	Seven point scale

Table 1

Additional attributes for the People, Process, and Technology components are shown in Figure 2 through Figure 4 in the appendix.

In addition to components, the project may have root attributes at the project level itself. Three examples of project root attributes are:

· Funding

· Number of scenarios

· Requirements volatility

A reminder: attributes are identified, defined, and added to the model because the value of the attribute has predictive power and influence on the successful outcome of the project.

Additional root project attributes can be seen in Figure 5 in the appendix.

As a final note, attributes can be combined to form a composite index. Composite indices have the potential to describe more about the project than an individual attribute.

Similar to modeling a sales forecast, using two attributes that have positive (non conflicting) correlation to sales will provide (via multiple regression analysis) more predictive behavior than a single attribute. For example, knowing GDP growth rate and the sales growth rate last period (time (N lag 1)) may very well provide substantially more predictive behavior than knowing just one of these attributes on their own. Now let’s think about the world of rules. Thus, rules on a composite index attribute (or multiple individual attributes) have the potential to carry more weight than a single attribute rule. An example of a composite index attribute is Communication index, represented by the composition of the following individual attributes:

Accessibility of requirements providers
Offshore component
Project size

In essence, the project context is the environment that surrounds the software development project under examination.

Project Model

A project model is the set of attributes with values that represent a particular project. A project model is the instantiation of a project context. As seen later in the document, the project model influences the various methodology models defined.

Finding project model values

Historical analysis

Typically, the project members, project manager, project stakeholders, project steering committee, or an expert independent 3^rd party can review the project and assign values to the attributes of the project context. Attributes whose values can range from “low to high” are intentionally given a seven (7) point scale to better enable later / after the fact statistical analysis on the attribute value and project outcome[4].

Future or Predictive analysis

Historical analysis is useful, as repetitive analysis of historical project context data can show trends, insight, causal effects, dependencies, and patterns in the project context model data. The IBM Corporation for years has had as part of their internal software development process a task called “causal analysis”. Six Sigma derived processes often have a similar task. These meetings take place after a project has come to an end. During these meetings, key project participants meet to discuss the items that went wrong during a project, and to discuss, as a group, what could have been done to prevent that problem from happening. Other organizations refer to this activity as root cause analysis. Whatever the name, these meetings often occur in an ad-hoc fashion, with a designated note taker. The participants discuss the problems, and brainstorm on ideas on how to have avoided that problem coming up in the first place. Meeting minutes are later distributed in an attempt to capture the decisions and allow those beyond the direct meeting participants to learn from the exchange.

While causal analysis meetings are useful, their ad-hoc nature to organizing the discussion of project problems make the meetings inconsistent. Imagine if the discussion could be organized around a set of project attributes and their values (project context model). With problems being framed by a common model, the language, approach, and analysis during these meetings could now be common. Discussion around the project context model attributes could incrementally add/modify to the set of known rules (best practices) for a project / project type.

Someone skilled in the art of software development, and methodology can do future or predictive analysis of a project context attribute values. Additionally, changing a project context model attribute can do predictive analysis. Simply re-run the recommendation engine, receiving an updated recommended methodology, list of fits, mis-fits, and recommendations and view the differences to determine the impact of that change in attribute value. By examining the list of mis-fits and recommendation, the user can examine the forces out of alignment with the best-fit methodology and gain insight into what will happen with that project should nothing be done.

Sample Project Model

The following is an example subset of a project model. The complete definition for project model attributes and their possible values can be seen in figures 2 - 5 in the appendix.

#=======================================================

# Project Context model

#=======================================================

# People attributes

People.NumberOfGeographicLocations=1

People.AccessibilityOfRequirementsProviders=5

People.size=14

People.CommunicationIndex=4

# Process attributes

Process.Deliverables=9

Process.PlannedBuildFrequency=3

Process.uniqueroles=4

Process.ProjectFlexibility=6

Process.MostFlexible=scope

Process.LeastFlexible=schedule

# Technology attributes

Project.technology.UserInterfaceCentric=4

Project.technology.Complexity=3

Project.technology.NumberOfThirdPartyInterfaces=2

Project.technology.Security=5

Project.technology.Scalability=4

# Root project attributes

Project.Funding=1000000

Project.Schedule=5

Project.scenarios=2

Project.screens=25

Project.RequirementsVolatility=5

Methodology Model

A methodology model (Figure 6) is a critically important extension of the Project Context concept. A methodology model is similar to a project context in that they share the same attribute members. However, a methodology model extends a project context by adding minimum and maximum values for an attribute, and substituting the attribute itself within the project context with a “mean” value in the methodology model. The range in values from low to high for an attribute is called the methodology model’s compatibility range for that given attribute. Think of a project context being an instantiation of a methodology model, substituting real project values in place of the mean values.

The values defined in the methodology model are important for rule processing. During steady state (a methodology has been selected and the project is underway), rules can be setup on the min and max values to notify the user that a movement across a min or max value (transition point) has occurred. These transition points represent input for the fit / misfit data output to the user when assessing a project that has already commenced. The transition point values within the Methodology models also make excellent candidates for attribute rules with a RHS of the Min or Max value for that methodology or an abstract representation of the methodology.

A rule will fire true if the value in the project context is within the compatibility range of the methodology model.

The min and max values within a methodology model are used for scoring and recommendation generation during the scoring process. Recommendations can be generated for project context attributes that are outside and/or significantly outside the compatibility range of a given methodology.

Rule Processing

What is a rule?

A rule or rule-set can represent a pattern in the Methodology pattern language. A rule has a Boolean output and maps two or more pieces of data along with an operation to see if the rule if the two pieces of data meet the condition of the operator.

Finding rules

Rules can be found in the following way:

· On hand statistical data.

· Heuristic data

Definitional rules

· Existing best practices

Defining rules

Software Project Context rules can be written on the following items:

Project Context data model

Project root attributes
People attributes
Process attributes
Technology attributes

Examples:

ProjectSize > 10 // number of developers

NumberOfTimeZones > 3 //

TechnologyComplexity > 5 // seven point scale

ProjectFunding > 1000000

Methodology Model. A methodology model has data elements for all the attributes in a project context, but adds two related data elements for each attribute in the project context:

Minimum compatibility value
Maximum compatibility value

In a methodology model, the project context attribute is substituted with a mean value.

Examples:

Project Context: Project.entities=28 // a real value

Methodology Model: Project.entities.min = 1 // min compatible value

Project.entities.mean = 60 // mean compatible value

Project.entities.max = 90 // max compatible value

Compatibility matrix:

A compatibility matrix shows all relevant project context attributes and their compatibility with a given methodology. The compatibility matrix identifies an important set of static information that is used by the methodology recommendation engine, but also forms the foundation for a Methodology selection pattern language.

Examples:

o Cell value: Specifying a row and column. (AccessibilityOfRequirementsProviders.low, RUP)

Two or more particular cell values: a comma delimited set of row,column pairs
Adjacent row cell value differences. Examines the state change of a given attribute by changing methodologies
Adjacent column cell value differences. Useful for attributes that span more than one row (low, medium, high). Can examine the state transition when the attribute value varies
Min/max value a row (across all methodologies)
Min./max value on the attribute column value (for those having multiple attribute value choices)

Rule-sets

A rule-set is one or more rules strung together by Boolean logic symbols. There can be simple and compound rule-sets. The following is an example of a Rule-set:

(ProjectSize > 10) and (NumberOfTimezones > 3)

Weighting of rules

Weighting of rules is an advanced topic. The exact scoring mechanism details are not outlined in this white paper. However, weighting of rules is touched upon in the “Advanced Topics” portion of the document.

Project Context methodology selection scoring

Example scoring process

The following data exists before the scoring process for a particular project context takes place:

Core rules and rule-sets
Project Context attribute values
Methodology compatibility matrix
Methodology models exist – one for each methodology (mean, min, and max values)

The following activities occur during the scoring process:

Core rules (representing best practices) are evaluated. Points and/or weighted points are awardeed
Each methodology model is examined relevant to the Project Context.
A set of fits and mis-fits for the project context relevant to each methodology is generated

After:

Recommended methodology is output (best score)
Agility score is output
Agility index is output
Set of fits and mis-fits for the recommended methodology is presented
Set of fits and mis-fits for other methodologies available on a demand basis
A list of recommendations regarding further configuration (tuning) of the Project Context attributes is output

Scoring output

o Recommended Methodology – the highest score (most compatible) methodology wins

o Agility Score – the agility point score of the project context when run against the agile core rules

o Agility index – a normalized value of the agility score that allows placement on the Agility Curve

o List of fits / mis-fits for that particular methodology associated. Even though a methodology was selected as “best fit” does not mean it is a perfect fit. The fits and mis-fits between the selected methodology and the project context being evaluated. The list of fits / mis-fits can be used to identify potential problems and for future “What if” attribute value change analysis.\

o List of fits / mis-fits for all methodology models evaluated (organized by Methodology). Having the list of all fits / mis-fits can be useful in spotting additional potential problems

o List of recommendations to project participants and stakeholders relevant to the project context

Scoring the Methodology model themselves

If a Project Context can be scored, so can a methodology model. After all, a Project Context is an instantiation of a methodology model.

In addition to the project context, one can also state that the same attribute definitions are applicable to a Methodology. Earlier, we stated that a project is an instantiation of a Methodology Model. If the Methodology has the people, process, and technology attributes, so does the Project Context. Time and time again, people ask, “why is this approach better than another”. The answer is that a methodology has a number of knobs that can be adjusted for a given project. The number of people on the project influences the Methodology delivery. The complexity of the technology influences the methodology delivery. The number of documented deliverables influences the delivery of the methodology. In essence, each attribute of the project impacts a given methodology. In addition to attribute values, the methodology also has minimum and maximum compatibility values for the given attribute. For example, people state that Extreme Programming is best suited for projects up to 15 people. In that embodiment, the minimum attribute value of size in the People component is 1. The maximum value is 15.

Because the Project Context is an instantiation of a Methodology Model, one can strip out the minimum and maximum values and set the mean value as the actual attribute value itself, as if it were an instantiated Project Context. The modified model data can then be put through the scoring process – assigning an agility score, agility index, and placement on the agility curve. Figures 13, 14, and 15 show the placement of the Methodology models on the agility curve.

If the min and max values are added back (extending the instantiated project context derived from the methodology model), one can perform repeated simulated project instantiations, with attributes settings randomly selected (normal distribution) between their min and max values. The summary of the results of such model simulation is shown in Figure 16.

Advanced topics

Compatibility Matrix

The fifth critical element of the patent is the definition of the Methodology Compatibility Matrix. The matrix shows all relevant project context attributes and their compatibility with a given methodology. The compatibility matrix identifies an important set of static information that is used by the methodology recommendation engine, but also forms the foundation for a Methodology selection pattern language. The compatibility shows alignment or non-alignment for a given project context attribute value and a methodology. Figure 11 shows a portion of an exemplary compatibility matrix.

The compatibility matrix is an important source of rule data for the methodology recommendation engine. Rules and rule-sets can be written for:

- A particular cell value

- Two or more particular cell values (rule-set)

- Adjacent row cell value differences (state transition)

- Adjacent column cell value differences (state transition)

- Min/Max on the row

- Min/Max on the column (for those attributes that have multiple columns).

For a 10x50 matrix, there are over 1800 potential rules that could be written.

Interpreting the Matrix

The following is a key for the symbols used in the matrix:

“+ +” – Strongly compatible

“+” - Compatible

“N” – Neither compatible nor incompatible – gives no signal / predictive power on the impact to the project

“-“ - Negative

“- -“ - Strongly negative

Project attributes (organized by component) represent rows in the matrix. Columns are represented by Methodologies. So, if I have 50 attributes, and 10 methodologies, there will be 500 cells in the matrix. Let’s examine a sample attribute having multiple rows as seen in the matrix snippet in Exhibit 1:

SunTone RUP SCRUM XP Waterfall

- Accessibility of requirements providers
	* low	-	-	- -	- -	N
	* medium	+	N	-	-	N
	* high	+	N	+	+	N

Exhibit 1

The first row tells us that low accessibility of requirements providers is a strong negative for both SCRUM and XP methodologies, a negative for the SunTone AM and RUP methodologies, and a neutral for the waterfall methodology. The second row tells us that medium accessibility to requirements providers is a negative for XP and SCRUM, a neutral for Waterfall and RUP, and a positive for SunTone AM. The third row tells us that high accessibility to requirements is a positive for SunTone AM, SCRUM, and XP, and a neutral for RUP and Waterfall. Methodology owners should provide their proposed entries for their column. As new methodologies and attributes are defined, new columns and rows can be added the compatibility matrix.

The Compatibility Matrix plays an essential role in the previously described statistical forecasting and simulation mechanisms for the scoring and recommendation engines.

SunTone RUP SCRUM XP Waterfall

- Accessibility of requirements providers
	* low	-	-	- -	- -	N
	* medium	+	N	N	-	N
	* high	+	N	+	+	N

Exhibit 2

As seen in Exhibit 2, as one traverses the column, think about a dial that exists on a radio labeled requirements provider accessibility. Let us assume that low is the baseline position for the dial. As the dial is turned to the right (one moves down the column), state transitions may take place for the cell value when moving to an adjacent cell in an adjacent row. Similarly, think about a radio dial labeled medium accessibility of requirements provider (a row on the matrix). As the dial is moved from setting to setting (methodology column to methodology column), state transitions may take place for the cell value when moving to an adjacent cell in an adjacent column. Significant state transitions are candidates for rules and subsequent capture of industry best practices. These state transitions, which are critically important for finding (mining) rules and subsequent patterns in the re-usable pattern-mining framework, will be discussed later in this paper.

Rule weighting

Weighting of rules are useful in two situations that come to mind:

Heuristic rules
Giving emphasis and/or breaking the tie

Die.net defines heuristic rule as a commonsense rule (or set of rules) intended to increase the probability of solving some problem

In the absense of defined rules, weighting can be used to further distinguish the “importance” of one rule versus another. Multiplying a rule (or rule-set) by less than one de-emphasizes the rule. Multiplying by greater than one makes the rule more important.

Examples:

Earlier it was mentioned that composite attributes could be more important than a regular attribute. A rule weighting could be used to simulate the extra importance/predictive power of the rule written on the composite attribute.

( TechnicalLeadership > 3 ) * 0.75

(CommunicationIndex > 4) * 1.5 // more than one as it is a composite attribute

The weighting can be used to influence the scoring mechanism – letting some rules add greater score, other rules add less ( or similarly, take away greater score, take away less score ).

Agility score

Agility in a project context may include one or more of, but not limited to, the following characteristics:

Assume simplicity (do not over-engineer)
Embrace change
Incremental change
Rapid feedback
Travel light (low number of artifacts)
Open communication
Continuous integration
Focus on people and communication rather than process and tools
Focus on working software rather than extensive documentation
Focus on customer collaboration rather than contract negotiation
Focus on responding to change rather than following a plan
Quick access to requirements source/validation/clarification

Some projects are very small and agile, while others are very large and cumbersome. A large number of projects fall in between. If one were to exclude the software projects being done by 1 or two people, one would see a distribution of multi-person project teams from Agile to non-Agile. This white-paper contends (although another distribution curve may be found later to be a better fit) that the distribution of multi-person project teams forms an Agility curve as seen in Figure 13. Repeating, the distribution of these projects, when measuring their agility (via an agility score of a project context), forms a curve approximating a normal distribution curve. Each “X” in the diagram represents an instantiation of a project context representing a real software project. Therefore, we can now apply concepts such as standard deviation and mean to software project Agility scores. Agility index calculation (resulting from the agility score and measured standard deviation) allows a project to be placed upon an Agility curve (Figure 14).

An agility index can be useful during a project assessment or project bidding timeframe. If the project is using a heavyweight methodology, but the agility index of the project context is 0.8 (or vice-versa), that is a quick and easy indicator to identify that there is a methodology mismatch with this project and significant opportunities exist to improve the project operations and cost.

Hecksel Agility Scale

Knowing the agility score of the project context is valuable. But comparing the scores of two or more projects is difficult. Is a 76 significantly different than an 82? The Agility index (Figure 14) has normalized values ranging from 0 to 1. Thus, someone skilled in the art of statistics could determine if a .67 was significantly different than a .84. But having values like .67 or .84 makes it difficult to effectively communicate the relative agility of a project based upon the agility score of the project context to a broad audience. A scale from 1 to 5 is much easier to understand and convey to business stakeholders, project managers, developers, and investors. The following table shows a mapping from agility index values to a 5-point agility scale – named the Hecksel agility scale, after it’s inventor[5].

Agility Index

Agility Scale

0.0 – 0.19

0.20 – 0.39

0.40 – 0.59

0.60 – 0.79

0.80 – 1.0

There is now a convenient, easy to communicate, and well understood mechanism to quickly convey the overall inherent agility of a software project.

Future Roadmap

Building the software product

First, we will examine a few legal statements regarding the software product definition within this white paper. The definition of the concepts contained within this white paper, Sun patent P8161, and the blueprint of a software product that provides guidance on methodology, methodology selection, and project assessment analysis is protected by patent. Software applications implementing the project context model described in this white paper to aide in software methodology, methodology selection, or project assessment are considered a violation of this patent. Only the filing party or a designated licensee can proceed with construction of such a software product for personal or business use. Now that we have that out of the way …

As described in this white paper, a systematic and automated approach to capturing the project context attribute values and performing an analysis on that context to:

Provide a best fit Methodology
Provide a set of fit / misfit recommendations for the recommended methodology, as well as (optionally) the fit / misfit data of each Methodology model as it was evaluated
Provide software project simulation to allow project simulation and what if analysis.

can be built.

UseCases for that application may include:

Entering Project Context attribute values for a given project
Finding a best fit methodology
View a best fit methodology recommendation detailed results
View methodology “fit” / “mis-fit” ( pattern / anti-pattern ) recommendations for the best fit methodology recommendation and the project context.
View cumulative “fit” / “mis-fit” data captured during scoring ( fits and mis-fits identified for all methodology model analysis ).
Configure the tool’s behavior through available options, such as “show all fit / misfit data”

Producing such an assessment product makes sense because of the additional business opportunities and revenue it could drive for Sun Services. Services personnel with such a product could:

· Enhance project-bidding effectiveness by fitting the right methodology to the right project (avoiding a one glove fits all approach).

· Introduction of a methodology assessment service. Software development is a tricky business. Customers need help and mentoring with software methodology. In addition, they know and have observed firsthand the pitfalls of adopting a one glove fits all approach to methodology ( a software development and corporate IT anti-pattern ). The software product described, when used by a trained user who can determine the project attributes by interviewing the customer, can dramatically improve their software development organization(s).

· Using the software product in-house within Sun Microsystems ( Sun Services, Sun IT, … ) to assess our own internal or customer delivery application development projects.

Figure 12 shows the legal rendition of such a software application as it appears in the patent “System and Method for Software Methodology Selection”.

In addition to using the product within Sun Microsystems, such an application would be interesting to companies in the business process and/or software development process automation space (Osellus software, bpml.org, IBM, …).

Rules versus Patterns

What is a Pattern?

Over time, there have been varying definitions of a pattern provided by the pattern community. Here are some of the past and present definitions:

“A pattern is the abstraction from a concrete form which keeps recurring in specific non-arbitrary contexts” (Dirk Riehle, Heinz Zullighoven)
“Each pattern is a three-part rule, which expresses a relation between a certain context, a problem, and a solution (Richard Gabriel)
“Named solution to a commonly recurring problem in a context which provides the understanding to tailor the solution to your variant of the problem” (John Vlissides)
“A solution to a commonly recurring problem given a context” (Brendan McCarthy)
“A documentation mechanism that captures general, tried and true solutions to common, recurring problems” (Steve Adolph and Paul Bramble derived from statements by Linda Rising)
Patterns are strategies. “Patterns as strategies help people thread their way through complex situations” ( Adolph and Bramble ). Is not the J2EE world a complex situation? The J2EE core patterns provide strategies for coping with complexity. Software methodology is also a complex world. Methodology patterns, and patterns derived from the Project Context, will provide strategies for success in the ambiguous and complex world of software development.

A pattern consists of the following sections: name; problem; context; forces; solution; examples; resulting context; rationale; related patterns; known uses.

Rules versus patterns

Are rules patterns? Are patterns rules? Clearly distinguishing the differences (and similarities) between rules and patterns helped this author create the methodology, methodology selection, and team composition pattern visions derived from the core project context patent. This author has been exposed to patterns since 1991 when working at IBM. Later, rules, rule management, and rules processing became important when inventing, architecting, and designing a web Personalization product called e.Monogram in 1997. e.Monogram attempted to capture, model, and predict a web-site visitor’s human behavior by mapping the behavior to a set of predictive rules. For example, the rule “LikesSports” would fire in the future as true if:

The user clicked on any of the banners that had the Sports metatag applied
Visited a page with the sports metatag applied
Answered an online survey answering positively to sports participating activities

Examples of rules are:

On Mondays, show Monday Night Football advertisements
Show Monday Night Football advertisements to males
Eat dinner after 6pm
Show socks banner to person who purchased shoes

Authors have distinguished rules from patterns in the following ways:

“A Rule says ‘do this’ without exploring the alternatives” (Adolph and Bramble)
“Neither are patterns rules you can apply mindlessly, the teaching component should counteract any such tendency” (Vlissides)
Each pattern is a rule which describes what you have to do to generate the entity which it defines (Alexander)

Rules and Patterns: Point Counterpoint

So what distinguishes a rule such as show Monday Night Football advertisements on Monday, or a rule-set ( show Monday Night Football advertisements to males on Monday ) from being a pattern? Let us examine some of the attributes and content of a pattern to distinguish the differences. As stated before, the attributes and content of a pattern are:

Name
Problem
Context
Forces
Solution
Examples
Resulting context
Rationale
Related patterns
Known uses

Let us examine the above one by one with respect to rules and patterns. First, everyday rules typically do not have names. For example, Jane Smith, a manager at ACME corp. has a rule that she buys pizza each Monday at 11:30am for her department. They eat together in a conference room and discuss various factors her team sees as requiring change within their company. Normally, one would not name a rule such as this. It would just happen. Each Monday, Jane would evaluate the situation, the Monday lunch rule would fire true, and she and her team sit down and have lunch together. But we could give this rule a name. Even for the fun of it. Perhaps one of the following names might fit:

DoLunch
DoLunchOnMondays
DoFood[6]

Rules are named in other situations too. When using a rules engine software product, it asks you to name all of your rules:

Rule Named Rule

If order > $50, perform credit check CreditCheck
If GeographicLocations >= 2 DistributedTeam
If PlaceOnTechnologyAdoptionCurve = 2 EarlyAdopter

So rules can have names. For example, showing Monday Night Football advertisements to males on Mondays could be named “LikeSports”. So rules typically do not have names (a difference), but can have them.

Second, let us examine “problem”. Certainly both rules and patterns have problems. The problem in each case above is maximizing viewer-ship, advertising revenue, and delivering a message to a target audience. Third, both rules and patterns deal with forces. The forces typically represent the attributes within the context. For the sports example, we have the forces of time, gender, past viewing history. Where rules and patterns begin to diverge is the level of detail on solution and further downstream content. The “LikesSports” rule above has a solution. Monday night football banner content is presented to all male viewers on Monday nights (the result of rule evaluation). However, the rule evaluation solution might be thought of as an instance solution. Patterns also deal with solutions but in a more abstract and detailed fashion. Patterns have greater documentation requirements than rules.

In the end, a rule can be thought of as an instance of a pattern execution. This makes good sense in that tasks like web personalization attempt to model human behavior patterns with rules. By looking at the behavior patterns, one creates rules that represent the execution of everyday behavior patterns. A context exists in each – the online media, the viewers, and the programming choices. Both the rule and pattern in this case have forces. Time is force (Monday night). Gender of the online visitor is a force. The attributes that one would write the personalization rules against are the forces. Both a rule and pattern share a solution, but differ in the depth of explanation. A rule might be said to have a result rather than a solution. For example, if a male visited the site on Monday, the result is the Monday night football advertisement appearing on the visitor’s screen. For the pattern, the solution is not only the result, but also an explanation

In the end, the discussion of rules and patterns can be a lot like the never ending debate “Tastes great” / “Less Filling”. Rules can often indicate the presence of a pattern and can be thought of as an instantiation of a pattern execution (rules imply patterns). Patterns often indicate rules. They are joined at the hip. The difference is the level of abstraction and amount of descriptive detail (context, resulting context, solution, rationale)

Re-usable pattern mining framework

In this white paper, forces are a set or subset of attributes that provide a context for moving towards one methodology or the other. Forces can be identified by large point scores for a relatively few number of attributes (or combination of attributes). Forces can also be identified by state changes in a compatibility matrix (minding the matrix)

The subsequent model can also be used for problem prediction – the identification of those forces that are out of alignment with the best-fit methodology choice.

A model of a software project was presented in this paper – an OO approach to a software project. Components, and subsequent attributes were defined. Rules can then be defined on those attributes to mine and capture patterns and best practices.

Now, what prevents a user from taking a similar approach to other contexts that exist in the world? Patterns have been applied to the technology community, the software development community in particular, for many years. Software developers and those in the technical community have formed a common communication language by referring to patterns when discussing a particular context or problem with one another. It is this author’s contention that the abstract summary naming mechanism for a complex problem is what software developers find appealing about patterns. Imagine if complexity could be abstracted away in subject areas / domains / contexts that software developers typically have problems communicating. Patterns on project management, building high performance teams, marketing, sales, leadership, … the gap between those in the technical community and the business community would be bridged by having a common language understood by both sides.

The re-usable pattern-mining framework consists of the following steps:

Select a context
Model the context

Identify components
Identify component attributes
Identify root context attributes

Is a compatibility matrix for similar approaches relevant? If so, adjacent cell values with differing values within the matrix are indicative of a change in state and indicates the possible presence of a pattern (mining the matrix). Step 3 is an optional step
Identify / create rules on the attribute values
Name the rules
Document patterns from the named rules (and optional matrix state transitions)

A companion white paper to this one has been created (Software Development Patterns) that the reader of this white paper may be interested. The paper documents patterns on the following areas:

Methodology
Methodology Selection
Software Team Composition

The Software Development Patterns paper leverages the project context ideas presented within this work.

The ubiquitous “About Box”

Who Am I?

Senior Java Architect

Sun Software Services

Sun Microsystems

david@davidhecksel.com

Factors that influenced the “critical factors” in the methodology evaluation and selection patent, and follow-on patterns

When I see an interesting idea or talk with someone with a unique perspective, I often think to myself:

· “How did the idea originate?”

· “What influenced the development of the idea”

Perhaps this line of thinking is influenced by exposure to Professor David Lindbergh, History of Science, University of Wisconsin-Madison[7]. Behind every idea, every step forward in the area of science, there were dozens of influencers who played critical roles doing their own work, making their own progress, and influencing the thinking of their peers and those who followed them. Roger Bacon is an excellent example of a key influencer in medieval science.

Here are some of the items that influenced my thoughts related to System and Method for Methodology Selection and follow-on pattern work:

· Observing first hand several large software project failures while employed at IBM in their Software Division 1985-1996

· The outstanding Object Oriented thinking approach instilled into me by IBM’s twelve week full time Object Oriented immersion program (Object Technology University)

· Exposure to the business world while at Duke University (MBA, ‘93), including organization behavior, strategy, and statistical forecasting

· Playing numerous roles in software development teams – from developer, designer, architect, manager, requirements analyst, CTO, and consultant

· Expanding upon my knowledge on personality and thinking types as a result of a child having ADD/ADHD (Attention Deficit Disorder). There have been many opportunities to learn about psychology, neurology, and learning differences as a result of a having a child with ADD/ADHD. There is a strong correlation between understanding learning differences, and understanding the various thinking styles of those in software development.

· Inventing, architecting, designing, and co-developing the e.Monogram web personalization server while CTO at Axtive Software. e.Monogram[8] had the capability to write rules (via a rule manager) on a web-site visitor’s site profile, click behavior on marketing objects, purchase history, system attributes (time, day of the week, …), survey question responses, and other customer defined rule categories. The exposure to rules and rule management working in the web personalization space certainly influenced the direction and content of the scoring and recommendation engines of “System and Method for Methodology Selection” patent.

· The observed customer preferences and patterns toward methodology selection while a part of the Sun Software Services team

· Participation in the Agile community mailing lists, online discussion forums, and presenting Agile methodology sessions at various technology conferences

· A passion to see the disparity and understanding gap between technology and business narrowed

Appendix

Project Context

Project Component Attribute Lists (People, Process, Technology, Project root)

The following attributes form the attributes of the People model. These attributes, when their values are examined, have predictive power over the outcome of a software development project. These attributes, when values are properly identified, can be used at project bidding time, project analysis/planning, what if decision analysis, statistical forecasting, and regression analysis.

People

Attribute Definition Scale

Geographic Locations	The number of geographic locations for all key and regular contributing project members	Numeric
TimeZones	The number of time zones for all key and regular contributing project members	Numeric
AccessibilityOf RequirementsProviders	Measured in latency of questions asked	Seven point scale
OffshoreComponent	Is there an off shore resource component with the project	Boolean
Development Offshore	The percent of development done offshore	Seven point scale
ReleaseManagerExperience	The experience ( measured in # of years ) of the person playing the role of Release Manager	1: 0 – 1 years 2: 1 – 2 years 3: 2 – 3 years 4: 3 – 4 years 5: 4 – 5 years 6: 5 – 6 years 7: 6 + years
ReleaseManagerExperience Diversity	The prior project diversity of the person playing the role of the Release manager	Seven point scale
ProjectManagerExperience	The experience ( measured in # of years ) of the person playing the role of Project Manager	1: 0 – 1 years 2: 1 – 2 years 3: 2 – 3 years 4: 3 – 4 years 5: 4 – 5 years 6: 5 – 6 years 7: 6 + years
SizeOfProject	The number of people on the project	Integer
Skill Level	The skill level of the composite team	Seven point scale
Communication Index	A composite attribute	Seven point scale
TechnicalLeadership	Leadership ability of the Technical Lead Developer	Seven point scale
ReleaseManagerLeadership	Leadership ability of the release manager	Seven point scale
SponsoringManagement Leadership	Leadership ability of the sponsoring manager	Seven point scale
LeadArchitectLeadership	Leadership ability of the Lead Architect	Seven point scale
Teamwork	The ability of the team to work together	Seven point scale
SeniorDeveloperRatio	the ratio of Senior ( Experienced diverse individuals who can Mentor, problem solve, achieve high productivity when needed, anticipate problems based on experience ) Developers on the team to non Senior Developers. Senior is a skill level and aptitude, not a job title.	1: 0/7 - 1/7 2: 1/7 - 2/7 3: 2/7 – 3/7 4: 3/7 – 4/7 5: 4/7 – 5/7 6: 5/7 – 6/7 7: 6/7 – 7/7
LeadArchitectExperience	The experience / prior project diversity experience of the lead architect ( a key role ).	1: 0 – 1 years 2: 1 – 2 years 3: 2 – 3 years 4: 3 – 4 years 5: 4 – 5 years 6: 5 – 6 years 7: 6 + years
LeadArchitectDiversity Experience	The experience / prior project diversity experience of the lead architect. Diversity experience measured by variety of prior software project experience - the same type of project over and over is much less interesting than a number of different types of project experience	Seven point scale
ProjectManagerDiversity Experience	The experience / prior project diversity experience of the lead architect. Diversity experience measured by variety of prior software project experience - the same type of project over and over is much less interesting than a number of different types of project experience	Seven point scale
PercentOffshore	If true on “offshore component”, the percentage of the development work done offshore	Percentage
ProjectManagerLeadership	Leadership ability of the project manager	Seven point scale

Figure 2

Process

The following attributes form the attributes of the Process model. These attributes, when their values are examined, have predictive power over the outcome of a software development project. These attributes, when values are properly identified, can be used at project bidding time, project analysis/planning, what if decision analysis, statistical forecasting, and regression analysis.

Attribute Definition Scale

Deliverables	How many, in what form, how many are kept up to date. Example deliverables may be a set of UseCases, a System Architecture Document, a Business Requirements Document, Test plan, … the gist of this attribute is to measure the “heavyweight” nature of document preparation, and ongoing maintenance ( some maintained, some a onetime delivery ). Depends on the project, also depends on the existing customer IT policies regarding documentation	Seven point scale There is a minimum of 1 – the Project Plan. Items which are formally delivered/reviewed count as .5, those that are delivered and require ongoing maintenance count as 1.0
NumberOfMandated Reviews	The Number of mandated artifact reviews	Seven point scale Integer ( minimum of 1 – the project plan ). Capped at 7I
PlannedBuildFrequency	The duration, measured in days, between product builds once some coding has begun	Seven point scale 1: >0 <= 1 day 2: >1 <= 2 days 3: >2 <= 3 days 4: >3 <= 4 days 5: >4 <= 5 days 6: > 5 <= 10 days 7: > > 10 days
PlannedUsageOfTools	The pervasiveness of tools used in the project for automation. Examples include project management, source management, defect tracking, …	Seven point scale
UniqueRoles	The number of different unique project roles ( requirements analyst, strategy, test, architect, project manager, technical facilitator, programmer, designer, techwriter, UI Designer, … )	Numeric
Format of requirements	Is there a constraint on the format of project requirements	None, UseCases, Stories, neutral
Environment Flexibility	Overall flexibility of the project environment	Seven point scale
Least flexible	Of the three answer choices, which is least flexible	Schedule, scope, or resources
Most flexible	Of the three answer choices, which is the most flexible	Schedule, scope, or resources
ProcessOwnerExperience WithMethodology	An array of answer choices – one for each Methodology considered. One answer choice for each Methodology considered	Numeric
ProjectManagerExperience WithMethodology	An array of answer choices – one for each Methodology considered. One answer choice for each Methodology considered	Numeric
ReleaseManagerExperience WithMethodology	An array of answer choices – one for each Methodology considered. One answer choice for each Methodology considered	Numeric
ArchitectureWorkflow	Is there a plan to have an Architecture Workflow?	Boolean
NeedForArchitecture WorkFlow	The perceived need for an architecture workflow?	Seven point scale
PlannedDailyMeetings	Are there planned daily meetings?	Boolean

Figure 3

Technology

The following attributes form the attributes of the Technology model. These attributes, when their values are examined, have predictive power over the outcome of a software development project. These attributes, when values are properly identified, can be used at project bidding time, project analysis/planning, what if decision analysis, statistical forecasting, and regression analysis.

Attribute Definition Scale

EstimatedPhysicalTiers	The number of estimated physical tiers in the final deployed system	Five point scale 1: one tier 2: two tiers 3: three tiers 4: four tiers 5: five+ tiers
UsesDistributedTechnology	Does the application utilize distributed technologies ( Corba, EJB, Messaging )	True / False
Reusability ( 1 )	Are there known advance re-usability requirements?	True / False
Reusability ( 2 )	Is this a Service Architecture? Are there requirements for re-usable components?	True / False
Scalability		Seven point scale
Availability		Seven point scale
Reliability		Seven point scale
Maintainability		Seven point scale
Security		Seven point scale
Complexity	The complexity of the technology used in the project	Seven point scale
NumberOf3rdPartyInterfaces	Number of 3^rd party interfaces / system integration points	Seven point scale 1: 0 2: 1 3: 2 4: 3 5: 4 6: 5 7: 6+
PlaceOnTechnology AdoptionCurve	Every technology has an adoption curve just like marketing adoption curves. Some projects have more than one technology used which would mean more than one technology adoption answer. The value for this attribute is the composite weighted adoption curve of the project. So - if the project is Java based, and 70% of the project is JSP/Servlet ( mainstream ) and 30% is Message Driven Beans ( Early adopter ), the answer is : (.7 * mainstream value) + (.3 * early adopter value) ---------------------------------------------- 2	Experimental=1 EarlyAdopter=2 Mainstream=3 Late Adopter=4
UICentric	How important is the user interface to the final delivered solution?	Seven point scale
EstimatedLogicalTiers	The number of estimated logical tiers in the final deployed system	Five point scale 1: one tier 2: two tiers 3: three tiers 4: four tiers 5: five+ tiers
EstimatedNumberOfBoxes	Number of boxes

Figure 4

Project root attributes

The following attributes are those identified at the root project level. These attributes, when their values are examined, have predictive power over the outcome of a software development project. These attributes, when values are properly identified, can be used at project bidding time, project analysis/planning, what if decision analysis, statistical forecasting, and regression analysis.

Attribute Definition Scale

Funding	The budget allocated to the project	Dollars
Schedule	The amount of time to complete the project – a release cycle	Months
Entities	Number of Entities / uniquely named objects in the application	Seven point scale
BusinessOwnerLeadership ControlFlexibility	Flexibility of business owner leadership control	Seven point scale
Scenarios	Number of Scenarios ( derived from UseCases ). A UseCase can have 1:N scenarios	1: 1 – 10 2: 11- 40 3: 41 - 100 4: 101 - 150 5: 151 - 200 6: 201 - 300 7: 301+
Screens	Number of screens for the solution of the project	1: 1 - 20 2: 21 – 50 3: 51 - 120 4: 121 - 200 5: 201 - 300 6: 201 - 425 7: 426+
RequirementsVolatility	How volatile the requirements are for the project	Seven point scale
Database Size ( number of Tables )	Number of Database tables there are in the database schema for the project	1: 1 - 25 2: 26 - 50 3: 51 – 75 4: 76 - 100 5: 101 - 125 6: 126 - 150 7: 151 - 175
Database Size ( number of records )	Number of rows in the tables with the largest number of rows	1: 1 – 10^2 2: 1 – 10^3 3: 1 – 10^4 4: 1 – 10^5 5: 1 – 10^6 6: 1 – 10^7 7: 1 – 10^8 +
Number of Entities	The number of Entities to be modeled and included in the new solution	1: 1 – 24 2: 25 - 49 3: 50 - 74 4: 75 - 124 5: 125 – 199 6: 200 - 299 7: 300+
Team Communication Technology	Measure of productivity / effectiveness of team communication	Seven point scale

Figure 5

Methodology Model

The following is a partial exemplary model file for eXtreme Programming – it does not contain all attributes. For the Methodology selection process, a model file exists for each Methodology participating in the evaluation process (attributes with min, mean, max values). Each methodology would have their own different min, mean, and max values. A rollup and comparison of those models, and how they can be used in the methodology selection process can be seen in Figure 11.

Project.funding.min=0

Project.funding.mean=1000000

Project.funding.max=3000000

Project.schedule.min=0

Project.schedule.mean=5

Project.schedule.max=15

Project.entities.min=1

Project.entities.mean=60

Project.entities.max=150

Project.people.geographiclocations.min=1

Project.people.geographiclocations.mean=1

Project.people.geographiclocations.max=1