Appendix C—Project Evaluation

C.1      Introduction

The MPO applied its goals and objectives as criteria to qualitatively evaluate the major infrastructure and capacity-adding highway projects that are in the Universe of Projects and Programs list (which had been sufficiently well defined to allow for analysis). Assessing how well projects would address the MPO’s goals and objectives helped the MPO identify priority projects for its major infrastructure program. Table C.1 shows the evaluated major infrastructure projects and Table C.2 provides a summary of the evaluated projects. A memorandum documenting the evaluation process also is included.

Technical Memorandum

DATE:     April 15, 2015

TO:         Boston Region MPO

FROM:    Sean Pfalzer and William Kuttner

RE:          Long-Range Transportation Plan Evaluation Criteria            

1          Overall Scoring System

For the 2040 LRTP, 38 projects classed as “major infrastructure” were evaluated by MPO staff. Based on these evaluations, MPO staff recommended inclusion of 13 of these projects in the LRTP, four of which were already programmed in the current TIP. Each of these 38 projects was given a numerical score, and this score to a large extent determined which projects were recommended for inclusion in the LRTP.

Each project was given a “high,” “medium,” or “low” rating in each of six rating categories. Expressing these ratings as numerical values of three, two, or one point respectively, the scores were summed resulting in a single numerical score for each project. While the scores could range from 18 points (six “high” scores) down to only 6 points, the 38 Major infrastructure projects ranged between 14 and 7 points.

Of the 13 projects recommended for inclusion in the LRTP, all had 11 or more points. Five projects had scores of at least 11 but were not recommended for inclusion because their costs were beyond the funding capabilities of the MPO regardless of their high scores.

1.1      Six Rating Categories

The rating categories were established based on the MPO’s adopted goals and focused on the primary goals addressed by Major Infrastructure projects:

• Safety
• System Preservation
• Capacity Management and Mobility (3 sub-categories include impacts to automobiles, buses, and pedestrian/bicyclists)
• Economic Vitality

The value of a project for each of these six areas was in turn characterized by a number of different factors. The evaluation criteria were grouped into the appropriate rating areas. MPO staff also identified or developed appropriate quantitative data and indices to help inform the scoring. As far as practicable, these criteria and indices had to be applied to all projects so that comparisons could be made between fundamentally dissimilar projects. More information on the evaluation criteria is presented below.

1.2      Role of Judgment in Determining a Score

Even with a reasonably complete set of planning-level evaluation data, the use of judgment is unavoidable in deciding which of the three scores to give projects for each of the six rating categories. There is, however, a structure within which judgment is applied. This process can be seen as a balancing of three factors:

• The needs in the proposed project area
• The criteria the proposed project addresses
• The impact a project can have in addressing the identified needs and advancing MPO goals

Of these three factors, the needs are perhaps best understood because they are derived from existing conditions. The configurations of proposed improvements are at this point conceptual and the extent and intensity of anticipated improvements can only be surmised.

Costs are not mentioned explicitly in these three factors. As a general rule, however, more costly projects will often have a larger impact. For instance, the safety and capacity of an obsolete intersection can be improved by rebuilding it to modern standards. In some instances constructing some kind of grade separation might be warranted. The costs will inevitably be greater but the benefits should also be greater. MPO staff accounted for cost to inform the safety rating for projects in order to compare projects across purpose and scale.

2          DEVELOPING SCORES IN EACH CATEGORY

One of the difficulties of scoring projects is choosing a scoring convention that will allow a valid comparison of dissimilar projects. Furthermore, fair and usable scoring conventions need to be developed separately for each of the six rating categories.

In developing a score it is important to consider the amount of improvement a project might be expected to achieve. This kind of project impact has been represented in this analysis by characterizing candidate projects by a very general “project concept.” The six project concepts used here are:

• Adding new grade separation
• Reconstructing of a major interchange
• Reconstructing of a minor interchange
• Significant widening of a road
• Minor widening of a road
• Reducing roadway capacity

The amount of improvement to safety and capacity in and near the project area will to some degree depend on the project concept.

The balance of this memo considers the rating categories individually. The indices, factors, and judgments that could result in a high, medium, or low score being assigned to a particular project are described and discussed for each of the six rating categories.

2.1      Safety

MPO staff maintains extensive databases of regional crash history, and these were used to assess the safety improvement needs for interchanges, express highways, and regional arterials. Crash history is measured using the “equivalent property damage only” index, abbreviated as EPDO. Crashes resulting in a fatality are given ten points, crashes resulting in injury five points, and property-damage-only crashes are given only one point. Given the relative infrequency of accidents, using the most recent three years of EPDO data in the candidate project areas gives a reasonable idea of the safety needs at that location.

Using the project-area EPDO values, staff developed indices that relate the crash history to project costs and projected users. Regional safety “hot spots” are identified by EPDO and may be addressed by candidate projects. EPDO related to specific modes and vehicle classes are also calculated and reviewed. These EPDO-based metrics include:

• Cost per EPDO (“cost effectiveness”): Estimated project cost divided by the EPDO value

• Average annual EPDO per 100,000,000 vehicles (“crash rate” or “risk”): Average annual EPDO value divided by average annual traffic volumes per 100,000,000 vehicles:          ((EPDO/3)/(AADT*330))*100,000,000

• EPDO concentrations
• Top 200 Crash Cluster Locations (Total EPDO)
• Highway Safety Improvement Program (HSIP) Cluster (Total EPDO)
• MPO-identified Truck Cluster (Truck-involved EPDO)
• HSIP Bicycle Cluster (bicycle-involved EPDO)
• HSIP Pedestrian Cluster (pedestrian-involved EPDO)

Choosing a score in the safety category requires comparing the severity of the safety problem with the improvement impact of the candidate project. As a general rule, the lower score of the two factors was the final score:

Low:         Either the need or the project benefit is low. Other factor may be higher

Medium: Either the need or the project benefit is medium. Other factor may be higher

High:        Both the safety need and project benefit is high

In assessing the project impact the project concept offers some general guidance:

• Adding new grade separation                              Low to medium
• Reconstructing of a major interchange              Medium to high
• Reconstructing of a minor interchange              Low to high
• Significant widening of a road                              Low to high
• Minor widening of a road                                       Low to high
• Reducing roadway capacity                                  Medium

An example of an improvement with a high safety benefit would be one that eliminates peak-period use of breakdown lanes on express highways or eliminates dangerous weaving movements at major interchanges.

2.2      System Preservation

MPO staff was able to use the state Road Inventory File and other sources to develop quantitative data for most candidate projects. The measured criteria include:

• Improves substandard pavement
• Pavement Condition (“fair” or “poor” pavement merit improvement)
• Number of lane-miles improved

• Improves substandard bridge
• Bridge Condition (structurally deficient or functionally obsolete merit improvement)
• Number of substandard bridges addressed

• Improves sidewalk infrastructure
• Number of sidewalk miles improved

• Improves bicycle facilities
• Number of bicycle lane-miles improved

• Improves emergency response or ability to respond to extreme conditions
• Improves access to an emergency support location
• Implements climate change adaptation strategies

The system preservation score was a judgment based on reviewing all the measured factors. An index that collapsed all the above factors into a single number was not used.

2.3      Capacity Management/Mobility: Autos

As part of the LRTP Needs Assessment, MPO staff analyzed several congestion measurements for both current and future conditions based on travel time, travel speed, and volume/capacity ratios to identify the worst bottleneck locations in the region. These MPO-identified bottleneck locations from the Needs Assessment were used to assess mobility-related needs of both express highways and regional arterials. Staff then assessed the impact of the project on managing capacity and improving mobility. The category scoring generally followed this pattern:

Low:         Project is not at an MPO-identified bottleneck location

            MPO-identified bottleneck would receive limited or no benefit

Medium: MPO-identified bottleneck will be addressed to a medium degree

            Non-bottleneck location is substantially improved

High:        MPO-identified bottleneck location is substantially improved

            New connection will improve mobility to a high degree

As in the safety category, levels of need and project benefit will vary across candidate projects, and judgment must be used to arrive at a score. A few examples can help illustrate this process.

Three projects that rated low, the I-290/I-495 interchange in Hudson, the Routes 126/135 junction in Framingham, and Middlesex Turnpike in Bedford were not MPO-identified bottleneck locations simply because other locations were significantly worse. Highland Avenue in Newton and Montvale Avenue in Woburn were also not MPO-designated bottlenecks. However, in these instances the improvements were considered great enough that the projects were given the score of “medium.” Complete reconstructions of old interchanges can also earn the medium ranking in the same way.

The heavily used I-93/I-95 interchange in Woburn is near the top of the list of regional bottlenecks. The improvements to the interchange and nearby roadways proposed as part of project reconstruction will result in a major improvement to regional traffic flow. At the other extreme is rebuilding the Boston Street overpass over the Lowell commuter rail line near the Wilmington-Woburn city line creates a completely new access corridor to an industrial area thereby earning a “high” rating.

2.4      Capacity Management/Mobility: Buses

To determine the bus mobility rating for congestion management,   MPO staff considered the level of bus service (MBTA and other local bus services) within the project area based on the number of routes and number of scheduled weekday bus trips. Then, using the auto mobility rating as the baseline, MPO staff assessed whether the bus service within area derived the same level of improvement as automobiles. The general scoring pattern for this category was:

Low:         No bus service within the project area or bus service will not be improved

                  Limited bus service and small to medium improvement for bus service

Some bus service within the project area but little bus service improvement

Medium:  Some bus service within the project area; and moderate service improvement

Significant bus service within the project area and smaller service improvement

High:        Significant project area bus service and significant service improvement

Judgment was required where projects seemed to fall between scoring levels. An example is the proposed Boardman Street grade separation. This is a severe arterial bottleneck causing delay to a large number of buses on some of the MBTA’s longest bus routes. However, because speeding traffic on this part of Route 1A would only shorten the bus travel times by a small percentage, a “medium” score for bus mobility has been assigned to the project.

The Route 3 widening would be a major improvement in a corridor that is considered to only have moderate congestion, as compared with its connecting highway, the Southeast Expressway. Few MBTA buses would benefit from the Route 3 widening, so the bus mobility score is “low.” Closer to downtown Boston, the Southampton Street interchange improvements would make a moderate impact at a location with severe congestion. Because more bus services would benefit, both auto and bus mobility improvements are rated “medium.”

2.5      Capacity Management/Mobility: Pedestrians and Bicycles

For the two non-motorized modes, the mobility issues relate primarily to the completeness and ease-of-use of the system of paths, sidewalks, and roads available for non-motorized travel. In evaluating candidate projects, MPO staff evaluates to what degree, if any, a project:

• Expands bicycle network, especially closing gaps in the system:
• Number of bicycle lane-miles added to the network
• Bay State Greenway Priority 100 project element
• High Priority Gap (flagged in the MPO’s Network Evaluation Study)

• Expands sidewalk network
• Number of sidewalks miles added to the network

• Improves transit access and intermodal connections
• Access to transit stations are improved for bicyclists and/or pedestrians

The project scores for this category reflect these benefit judgments:

Low:         Bicycle and pedestrian facilities are not applicable to the project

                  Bike/pedestrian facilities will be expanded to a low degree

Medium:  Bike/pedestrian facilities will be expanded to a moderate degree

                  Project meets healthy transportation policy directive standards

High:        Bike/pedestrian facilities will be expanded to a high degree

                  Project exceeds healthy transportation policy directive standards

2.6      Economic Vitality

While any major transportation improvement can be expected to contribute to economic vitality, the ratings in this category reflect to what degree the improvements support the land use objectives embraced by the MPO. A candidate project can support these objectives if it:

• Provides access to target development area
• Vehicle, transit, bicycle, or pedestrian access improvements

• Serves existing area of concentrated development
• High population and employment density for type of community

• Facilitates new development
• Transportation project is tied to new development proposals

The project scores for this category reflect these benefit judgments:

Low:         Project does not provide access to a targeted development area or area of concentrated development.

Medium:  Project provides access to a targeted development area or area of concentrated development to a moderate degree or facilitates economic development

High:        Project provides access to a targeted development area or area of concentrated development to at least a moderate degree, and it facilitates new development.

SP/WK/wk