SG 2 Provide Measurement Results
Measurement results that address identified information needs and objectives are provided. The primary reason for doing measurement and analysis is to address identified information needs and objectives. Measurement results based on objective evidence can help to monitor performance, fulfill contractual obligations, make informed management and technical decisions, and enable corrective actions to be taken.
SP 2.1 Collect Measurement Data
Obtain specified measurement data. The data necessary for analysis are obtained and checked for completeness and integrity.
Typical Work Products
1. Base and derived measurement data sets
2. Results of data integrity tests
Subpractices
1. Obtain the data for base measures.
Data are collected as necessary for previously used as well as for newly specified base measures. Existing data are gathered from project records or from elsewhere in the organization. Note that data that were collected earlier may no longer be available for reuse in existing databases, paper records, or formal repositories.
2. Generate the data for derived measures.
Values are newly calculated for all derived measures.
3. Perform data integrity checks as close to the source of the data as possible. All measurements are subject to error in specifying or recording data. It is alwaysbetter to identify such errors and to identify sources of missing data early in the measurement and analysis cycle. Checks can include scans for missing data, out-of-bounds data values, and unusual patterns and correlation across measures. It is particularly important to do the following:
Test and correct for inconsistency of classifications made by human judgment(i.e., to determine how frequently people make differing classification decisionsbased on the same information, otherwise known as "inter-coder reliability").
· Empirically examine the relationships among the measures that are used to calculate additional derived measures. Doing so can ensure that important distinctions are not overlooked and that the derived measures convey their intended meanings (otherwise known as "criterion validity").
SP 2.2 Analyze Measurement Data
Analyze and interpret measurement data. The measurement data are analyzed as planned, additional analyses are conducted as necessary, results are reviewed with relevant stakeholders, and necessary revisions for future analyses are noted.
Typical Work Products
1. Analysis results and draft reports
Subpractices
1. Conduct initial analyses, interpret the results, and draw preliminary conclusions. The results of data analyses are rarely self evident. Criteria for interpreting theresults and drawing conclusions should be stated explicitly.
2. Conduct additional measurement and analysis as necessary, and prepare results for presentation. The results of planned analyses may suggest (or require) additional, unanticipated analyses. In addition, they may identify needs to refine existing measures, to calculate additional derived measures, or even to collect data for additional primitive measures to properly complete the planned analysis. Similarly, preparing the initial results for presentation may identify the need for additional,unanticipated analyses.
3. Review the initial results with relevant stakeholders.It may be appropriate to review initial interpretations of the results and the way in which they are presented before disseminating and communicating them more widely. Reviewing the initial results before their release may prevent needless misunderstandings and lead to improvements in the data analysis and presentation. Relevant stakeholders with whom reviews may be conducted include intendedend users and sponsors, as well as data analysts and data providers
4. Refine criteria for future analyses. Valuable lessons that can improve future efforts are often learned from conducting data analyses and preparing results. Similarly, ways to improve measurement specifications and data collection procedures may become apparent, as may ideas for refining identified information needs and objectives.
SP 2.3 Store Data and Results
Manage and store measurement data, measurement specifications, and analysis results. Storing measurement-related information enables the timely and cost effectivefuture use of historical data and results. The information also is needed to provide sufficient context for interpretation of the data,measurement criteria, and analysis results. Information stored typically includes the following:
· Measurement plans
· Specifications of measures
· Sets of data that have been collected
· Analysis reports and presentations
The stored information contains or references the information needed to understand and interpret the measures and assess them for reasonableness and applicability (e.g., measurement specificationsused on different projects when comparing across projects).
Data sets for derived measures typically can be recalculated and need not be stored. However, it may be appropriate to store summaries based on derived measures (e.g., charts, tables of results, or report prose). Interim analysis results need not be stored separately if they can be efficiently reconstructed. Projects may choose to store project-specific data and results in aproject-specific repository. When data are shared more widely acrossprojects, the data may reside in the organization’s measurement repository.
Typical Work Products
1. Stored data inventory
Subpractices
1. Review the data to ensure their completeness, integrity, accuracy,and currency.
2. Make the stored contents available for use only by appropriategroups and personnel.
3. Prevent the stored information from being used inappropriately.
Examples of ways to prevent inappropriate use of the data and related information include controlling access to data and educating people on the appropriate use of data.
Examples of inappropriate use include the following:
· Disclosure of information that was provided in confidence
· Faulty interpretations based on incomplete, out-of-context, or otherwise misleading information
· Measures used to improperly evaluate the performance of people or to rank projects
· Impugning the integrity of specific individuals
SP 2.4 Communicate Results
Report results of measurement and analysis activities to all relevant stakeholders. The results of the measurement and analysis process arecommunicated to relevant stakeholders in a timely and usable fashionto support decision making and assist in taking corrective action.Relevant stakeholders include intended users, sponsors, data analysts,and data providers.
Typical Work Products
1. Delivered reports and related analysis results
2. Contextual information or guidance to aid in the interpretation of analysis results
Subpractices
1. Keep relevant stakeholders apprised of measurement results on a timely basis. Measurement results are communicated in time to be used for their intended purposes. Reports are unlikely to be used if they are distributed with little effort tofollow up with those who need to know the results. To the extent possible and as part of the normal way they do business, users of measurement results are kept personally involved in setting objectives and deciding on plans of action for measurement and analysis. The users are regularly kept apprised of progress and interim results.
2. Assist relevant stakeholders in understanding the results.Results are reported in a clear and concise manner appropriate to themethodological sophistication of the relevant stakeholders. They are understandable, easily interpretable, and clearly tied to identified information needs and objectives. The data are often not self evident to practitioners who are not measurement experts. Measurement choices should be explicitly clear about the following:
How and why the base and derived measures were specified
How the data were obtained
How to interpret the results based on the data analysis methods that were used
How the results address their information needs
Examples of actions to assist in understanding of results include the following:
· Discussing the results with the relevant stakeholders
· Providing a transmittal memo that provides background and explanation
· Briefing users on the results
· Providing training on the appropriate use and understanding of measurement results
