1 of 35 pages
A Practical Approach to Defining Useful Measures
Being effective in today’s business and project environments requires that decisions be made quickly based on the best information available. Organizations face decisions like:
• How many changes to the requirements can a project handle without slipping?
• What improvement initiative is providing the best result?
• Which projects should be funded?
• Will we be shipping the products we promised on time this quarter?
Project teams are concerned about issues such as their ability to meet schedules with all customer requirements satisfied. In this presentation, participants will learn a practical and structured method for defining measures that help with their decision-making.
0 Comments on this document
Document Transcript:
Facilitating your process
journey «
A Practical Approach to Defining
Useful Measures
Houston Texas Chapter of the IIBA
Houston, TX
August 7, 2008AgendaF]o] Ç } v
Measurement and Analysis Process Overview
Plan Measurement
Perform Measurement
Evaluate Measurement
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 2F]o] Ç } v
Measurement Process Model
User Feedback
Technical and
Objectives Management
and Issues
Processes Analysis Results
Core Measurement Activities
Measurement
Plan
Plan Perform
Measurement New Issues Measurement
Analysis Reports
and Performance
Improvement Measures
Actions Evaluate
Measurement
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 3F]o] Ç } v
Role of Measurement
The purpose of measurement is to provide the proper objective information needed to
make informed decisions that affect the business and/or project outcome
Supports making Supports tracking Supports Enables effective
Aids early problem informed decisions of progress to evaluating process communication of
identification and justifying the achieve specific and project process and project
decisions goals and performance performance
objectives results
M (( ]À ]uo u v } } Pv]Ì]}v µ v Zv }i ]À
and when integrated with existing technical and management activities that define a project
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 4F]o] Ç } v
Fundamental Measurement Principles - 1
1. Measurement and Analysis is a process
2. Information needs define the measurement objectives which
in turn define the measures that support the information needs
3. Processes define the measures that support the
measurement objectives
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 5F]o] Ç } v
Fundamental Measurement Principles - 2
P} }] ]vµu v
collection
5. Measures are analyzed to ensure that only necessary data
are collected to address the measurement objectives
6. Measures are used to aid the decision making process
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 6F]o] Ç } v
Fundamental Measurement Principles - 3
7. Analysis results are interpreted within the context of other
information and data (integrated analysis)
8. Measurements provide objective information to facilitate
effective communication
9. Measurement and Analysis techniques are applied to the
Measurement and Analysis process
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 7F]o] Ç } v
Measurement and Analysis Process
Technical and User Feedback
Principle #1 Objectives Management
Measurement and Issues Analysis Results
Processes
and Analysis is a
process
Core Measurement Activities
Measurement
Plan Plan Perform
Measurement Measurement
New Issues
Analysis Reports
Improvement and Performance
Actions Evaluate Measures
Measurement
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 8F]o] Ç } v
Issue-Driven Measurement
Principle #2
Information needs define the
Information
measurement objectives which in
Needs
turn define the measures that
support the information needs
Objectives,
Measures
Issues
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 9F]o] Ç } v
Measurement Framework
Goals and Objectives
Corporate
Performance Measures
Focus
Senior Management Customers
Measurement Repository
Process Focus Process Measures Estimates
A B C Quality data
Process Data Peer Review data
Customer Satisfaction data
Etc.
Project
/Product Focus
Project 1 Project 2 Project 3 Project 4 Project /
Product Data
Historical Measures
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 10F]o] Ç } v
Measurement and the Process
Principle #3
–Process characteristics will drive: Processes define the
–what can be measured measures that support the
measurement objectives
–how measures are generated and captured
–Measurement should be a by-product of process execution
Coding Activity Testing Activity
Example Base 1. lines of code 1. # of defects found
Measures 2. complexity 2. defect type
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 11AgendaF]o] Ç } v
Measurement and Analysis Process Overview
Plan Measurement
Perform Measurement
Evaluate Measurement
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 12F]o] Ç } v
Plan Measurement
Describe
Identify and information needs Design the measures
prioritize and associated to address each
objectives/issues measurable information need
concepts
Develop the Build the
measurement measurement plan,
procedures to integrating
collect and organize measurement into
the data project processes
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 13F]o] Ç } v
Identify and Prioritize Objectives/Issues
–Each project and organization has unique information
needs
–When identifying issues, consider sources such as
–constraints and assumptions
–project or organization characteristics
domain, technology
people and skills
product features and acceptance criteria
cost, schedule, and milestones
–risks
–past experience of the team
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 14F]o] Ç } v
Describe the Information Need
–For each top rated objective or issue, describe the information need
–who needs to know what informationin order to make what decision
–Results from measurement must
–be useable by the person or group (who)
–answer the need (know what information)
–enable the action (make what decision)
–Describe a measurable concept for each
need
–what data might be used
–gathered and analyzed at what times
–reported in what ways
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 15F]o] Ç } v
Example of a Common Information Need
–Issue: Project schedule appears to be very aggressive
Information Needs:
–The Project Manager and team membersneed to regularly determine which
tasks are late, to be able to make corrections in the plan.
–The Project Managerneeds to know the cumulative effects of task slippage
during design, code, and test, to know when a significant problem occurs.
Collect and analyze data
at a level of detail
sufficient to identify and
isolate problems.
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 16F]o] Ç } v
Objectives and Information Needs
ObjectiveRequirements need to be managedbetter
since volatile requirements are causing
projects to fail.
Information Need Measurable Concept
Example: The Project Manager needs to Example: At the end of the week, the
know the number of requirements changes Requirements Analyst reports to the Project
that the project can handle without Manager the current requirements stability,
additional schedule or budget increases. total and by type of change. The Project
Manager investigates the requirements
growth for the week and the month and
modifies the plan as appropriate.
AS TEAMS,DEVELOP AN EXAMPLE
INFORMATION NEED AND CONCEPT USING
YOUR HANDOUT
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 17F]o] Ç } v
Levels of a Measurement Construct
–Measurement construct captures a chosen approach
–identifies levels of measures that address the information need
Attribute
Base Measure one or more
collections
of Measures
are used to
Derived Measure build an
Information
Product
Indicator
Information Product
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 18F]o] Ç } v
Leverage Existing Systems
Effort, Schedule, Plans, & Actuals
History, Effort/
Skills Schedule Project Effort Actuals
Measurement Estimation Tool Management
Estimates
Repository System
Milestones,
Sizing Estimates, Actuals, Status Tasks,
Characteristics
Assignments
Counts,
Requirements
Effort Time
Management Time
Project Spent
System Reporting
ABC System
Defects,
Rework
Module and Budget,
Status Labor
Change Size Expenditures
Costs
Defect Configuration
Financial
Reporting Management
System
System System
Cost, Budget, & Actuals
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 19F]o] Ç } v
What Can You Leverage?
Tools Available Information Needs It Additional Needs it
? Now Serves Might Serve
Estimation Models/Methods
Requirements Management
Time Reporting
Project Management
Cost Accounting
Defect Tracking
Code Counting and/or Code Analysis
Configuration Management
Test Automation
Others?
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 20F]o] Ç } v
Integrate Measurement with the Process
Use already
Build Identify established Use Quality Institutional-
Build into necessary process meetings Assurance ize
process collection owner for Provide and briefings function to collection,
documenta- and analysis collection training to present verify analysis, and
tion tools and analysis and discuss process is use
results followed
Principle #4
P} }] o vµu v oo} }oo ]}v
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 21AgendaF]o] Ç } v
Measurement and Analysis Process Overview
Plan Measurement
Perform Measurement
Evaluate Measurement
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 22F]o] Ç } v
Perform Measurement
Collect and process
base and derived
measures
Make decisions and Generate indicators
take action
Analyze Data
These steps occur continuously throughout a project.
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 23F]o] Ç } v
Key Concepts
Data need to be captured as close to their creation point as possible
Raw data need to be verified before analysis
Indicators are the building blocks of analysis
Analysis is an investigative activity - indicator generation should be flexible
enough to satisfy known information needs and to dynamically uncover
root causes of problems
All analysis involves comparing plan/baseline/expected with actual results
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 24F]o] Ç } v
Indicators
–At the reporting periods planned, generate and report
the indicators
–produce standard indicators on regular basis, as planned;
special indicators as needed
Unit Test Status
–analyze indicators to localize problems 140
120
–identify new indicators that 100
80
respond to new questions 60
40 Plan
20
–collect sets of related 0 Actual
indicators
# of Units Completing Unit Test
Project Status Date
Project: Date Prepared: 1
OLDMS Oct 2009
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 25F]o] Ç } v
Analyze Data
–Analyze the indicators systematically, to assess the
status of known issues and to identify new issues
–Have the analysis done by someone independent of
those who produce and use the data
–While doing the analysis
–identify problems Principle #5
Measures are analyzed to
–assess problem impact ensure that only necessary
–forecast outcome data are collected to
address the measurement
–evaluate alternatives objectives
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 26F]o] Ç } v
Integrated Analysis Example for the Information Need:
Are we ready to deliver?
Requirements Stability
Problem Report Status
800
700
600 600
500
400 400
300
200 200
Plan 100
Discovered
0 0
Actual
Closed
Number of Requirements
Project Status Date Project Status Date
Number of Problem Reports
Project: Date Prepared: 1 Oct
Project: Date Prepared: 1
OLDMS 2009
OLDMS Design Progress Oct 2009
140
120
100
80 Principle #6
60
40
Measures are used to aid
20
Plan
0
the decision making
Actual
process
# of Units Completed
Project Status Date
Project: OLDMS Date Prepared: 1 Oct
2009
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 27F]o] Ç } v
Reporting Results
–Use a systematic analysis process as the basis for
reporting results:
–identify problems Principle #7
–assess problem impact Analysis results are interpreted within
–forecast outcome the context of other information and
–evaluate alternatives data (integrated analysis)
–Example:
Problem Impact Outcome Alternatives
Test planning not Test Plan - Medium Testing will be late or Shift Test Planning to
started Development - Low quality will suffer if Analysts
Development is Staffing - High ovv]vP Test Component C last
slightly behind in the next 2 weeks Extend schedule
schedule with If staffing lag L - (( v}v
Component C continues or worsens, project-related
Staffing is below project will be late commitments
planned levels
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 28F]o] Ç } v
Make Decisions and Take Action
–The measurement results must be clearly Principle #8
understood by the decision makers Measurements provide objective
information to facilitate effective
communication
–Make results available openly, to all those affected
–Action must be taken to realize any benefit from measurement
–Desired actions may not be possible -may have to optimize within existing
constraints
Principle #9
Measurement and Analysis techniques are applied to the
Measurement and Analysis process
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 29AgendaF]o] Ç } v
Measurement and Analysis Process Overview
Plan Measurement
Perform Measurement
Evaluate Measurement
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 30F]o] Ç } v
Evaluating Measurement
Evaluate the
Evaluate the measures
measurement process
Update the experience Identify and implement
base improvements
Like the products and processes it serves, the measurement process periodically
needs review and analysis, too
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 31F]o] Ç } v
Summary
Planning identify key issues and information needs
specify measures that assist decision-making
Measurement
collect measures and verify the data
Performing
analyze the data and generate indicators
Measurement
use the measurement results for decision-making
Evaluating
review the products and process of measurement,
to identify improvements
Measurement
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 32F]o] Ç } v Questions
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 33F]o] Ç } v
PPQC Services
Consulting
CMMI
Software Engineering
Systems Engineering
Process Improvement
Appraising
SCAMPI A, B, C
Gap Analysis
Training
CMMI/Process Improvement
Action Planning Workshops
Measurement and Analysis
Process Area Specific Training
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 34F]o] Ç } v
Contact Information
Address: Henry Schneider
President/Senior Principal Consultant
Process and Product Quality Consulting, LLC
2111 Heather Green
Houston, TX USA 77062
Phone: 281-218-6682
E-mail: henry@ppqc.net
Web Site: www.ppqc.net
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 35
journey «
A Practical Approach to Defining
Useful Measures
Houston Texas Chapter of the IIBA
Houston, TX
August 7, 2008AgendaF]o] Ç } v
Measurement and Analysis Process Overview
Plan Measurement
Perform Measurement
Evaluate Measurement
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 2F]o] Ç } v
Measurement Process Model
User Feedback
Technical and
Objectives Management
and Issues
Processes Analysis Results
Core Measurement Activities
Measurement
Plan
Plan Perform
Measurement New Issues Measurement
Analysis Reports
and Performance
Improvement Measures
Actions Evaluate
Measurement
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 3F]o] Ç } v
Role of Measurement
The purpose of measurement is to provide the proper objective information needed to
make informed decisions that affect the business and/or project outcome
Supports making Supports tracking Supports Enables effective
Aids early problem informed decisions of progress to evaluating process communication of
identification and justifying the achieve specific and project process and project
decisions goals and performance performance
objectives results
M (( ]À ]uo u v } } Pv]Ì]}v µ v Zv }i ]À
and when integrated with existing technical and management activities that define a project
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 4F]o] Ç } v
Fundamental Measurement Principles - 1
1. Measurement and Analysis is a process
2. Information needs define the measurement objectives which
in turn define the measures that support the information needs
3. Processes define the measures that support the
measurement objectives
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 5F]o] Ç } v
Fundamental Measurement Principles - 2
P} }] ]vµu v
collection
5. Measures are analyzed to ensure that only necessary data
are collected to address the measurement objectives
6. Measures are used to aid the decision making process
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 6F]o] Ç } v
Fundamental Measurement Principles - 3
7. Analysis results are interpreted within the context of other
information and data (integrated analysis)
8. Measurements provide objective information to facilitate
effective communication
9. Measurement and Analysis techniques are applied to the
Measurement and Analysis process
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 7F]o] Ç } v
Measurement and Analysis Process
Technical and User Feedback
Principle #1 Objectives Management
Measurement and Issues Analysis Results
Processes
and Analysis is a
process
Core Measurement Activities
Measurement
Plan Plan Perform
Measurement Measurement
New Issues
Analysis Reports
Improvement and Performance
Actions Evaluate Measures
Measurement
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 8F]o] Ç } v
Issue-Driven Measurement
Principle #2
Information needs define the
Information
measurement objectives which in
Needs
turn define the measures that
support the information needs
Objectives,
Measures
Issues
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 9F]o] Ç } v
Measurement Framework
Goals and Objectives
Corporate
Performance Measures
Focus
Senior Management Customers
Measurement Repository
Process Focus Process Measures Estimates
A B C Quality data
Process Data Peer Review data
Customer Satisfaction data
Etc.
Project
/Product Focus
Project 1 Project 2 Project 3 Project 4 Project /
Product Data
Historical Measures
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 10F]o] Ç } v
Measurement and the Process
Principle #3
–Process characteristics will drive: Processes define the
–what can be measured measures that support the
measurement objectives
–how measures are generated and captured
–Measurement should be a by-product of process execution
Coding Activity Testing Activity
Example Base 1. lines of code 1. # of defects found
Measures 2. complexity 2. defect type
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 11AgendaF]o] Ç } v
Measurement and Analysis Process Overview
Plan Measurement
Perform Measurement
Evaluate Measurement
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 12F]o] Ç } v
Plan Measurement
Describe
Identify and information needs Design the measures
prioritize and associated to address each
objectives/issues measurable information need
concepts
Develop the Build the
measurement measurement plan,
procedures to integrating
collect and organize measurement into
the data project processes
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 13F]o] Ç } v
Identify and Prioritize Objectives/Issues
–Each project and organization has unique information
needs
–When identifying issues, consider sources such as
–constraints and assumptions
–project or organization characteristics
domain, technology
people and skills
product features and acceptance criteria
cost, schedule, and milestones
–risks
–past experience of the team
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 14F]o] Ç } v
Describe the Information Need
–For each top rated objective or issue, describe the information need
–who needs to know what informationin order to make what decision
–Results from measurement must
–be useable by the person or group (who)
–answer the need (know what information)
–enable the action (make what decision)
–Describe a measurable concept for each
need
–what data might be used
–gathered and analyzed at what times
–reported in what ways
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 15F]o] Ç } v
Example of a Common Information Need
–Issue: Project schedule appears to be very aggressive
Information Needs:
–The Project Manager and team membersneed to regularly determine which
tasks are late, to be able to make corrections in the plan.
–The Project Managerneeds to know the cumulative effects of task slippage
during design, code, and test, to know when a significant problem occurs.
Collect and analyze data
at a level of detail
sufficient to identify and
isolate problems.
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 16F]o] Ç } v
Objectives and Information Needs
ObjectiveRequirements need to be managedbetter
since volatile requirements are causing
projects to fail.
Information Need Measurable Concept
Example: The Project Manager needs to Example: At the end of the week, the
know the number of requirements changes Requirements Analyst reports to the Project
that the project can handle without Manager the current requirements stability,
additional schedule or budget increases. total and by type of change. The Project
Manager investigates the requirements
growth for the week and the month and
modifies the plan as appropriate.
AS TEAMS,DEVELOP AN EXAMPLE
INFORMATION NEED AND CONCEPT USING
YOUR HANDOUT
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 17F]o] Ç } v
Levels of a Measurement Construct
–Measurement construct captures a chosen approach
–identifies levels of measures that address the information need
Attribute
Base Measure one or more
collections
of Measures
are used to
Derived Measure build an
Information
Product
Indicator
Information Product
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 18F]o] Ç } v
Leverage Existing Systems
Effort, Schedule, Plans, & Actuals
History, Effort/
Skills Schedule Project Effort Actuals
Measurement Estimation Tool Management
Estimates
Repository System
Milestones,
Sizing Estimates, Actuals, Status Tasks,
Characteristics
Assignments
Counts,
Requirements
Effort Time
Management Time
Project Spent
System Reporting
ABC System
Defects,
Rework
Module and Budget,
Status Labor
Change Size Expenditures
Costs
Defect Configuration
Financial
Reporting Management
System
System System
Cost, Budget, & Actuals
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 19F]o] Ç } v
What Can You Leverage?
Tools Available Information Needs It Additional Needs it
? Now Serves Might Serve
Estimation Models/Methods
Requirements Management
Time Reporting
Project Management
Cost Accounting
Defect Tracking
Code Counting and/or Code Analysis
Configuration Management
Test Automation
Others?
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 20F]o] Ç } v
Integrate Measurement with the Process
Use already
Build Identify established Use Quality Institutional-
Build into necessary process meetings Assurance ize
process collection owner for Provide and briefings function to collection,
documenta- and analysis collection training to present verify analysis, and
tion tools and analysis and discuss process is use
results followed
Principle #4
P} }] o vµu v oo} }oo ]}v
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 21AgendaF]o] Ç } v
Measurement and Analysis Process Overview
Plan Measurement
Perform Measurement
Evaluate Measurement
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 22F]o] Ç } v
Perform Measurement
Collect and process
base and derived
measures
Make decisions and Generate indicators
take action
Analyze Data
These steps occur continuously throughout a project.
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 23F]o] Ç } v
Key Concepts
Data need to be captured as close to their creation point as possible
Raw data need to be verified before analysis
Indicators are the building blocks of analysis
Analysis is an investigative activity - indicator generation should be flexible
enough to satisfy known information needs and to dynamically uncover
root causes of problems
All analysis involves comparing plan/baseline/expected with actual results
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 24F]o] Ç } v
Indicators
–At the reporting periods planned, generate and report
the indicators
–produce standard indicators on regular basis, as planned;
special indicators as needed
Unit Test Status
–analyze indicators to localize problems 140
120
–identify new indicators that 100
80
respond to new questions 60
40 Plan
20
–collect sets of related 0 Actual
indicators
# of Units Completing Unit Test
Project Status Date
Project: Date Prepared: 1
OLDMS Oct 2009
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 25F]o] Ç } v
Analyze Data
–Analyze the indicators systematically, to assess the
status of known issues and to identify new issues
–Have the analysis done by someone independent of
those who produce and use the data
–While doing the analysis
–identify problems Principle #5
Measures are analyzed to
–assess problem impact ensure that only necessary
–forecast outcome data are collected to
address the measurement
–evaluate alternatives objectives
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 26F]o] Ç } v
Integrated Analysis Example for the Information Need:
Are we ready to deliver?
Requirements Stability
Problem Report Status
800
700
600 600
500
400 400
300
200 200
Plan 100
Discovered
0 0
Actual
Closed
Number of Requirements
Project Status Date Project Status Date
Number of Problem Reports
Project: Date Prepared: 1 Oct
Project: Date Prepared: 1
OLDMS 2009
OLDMS Design Progress Oct 2009
140
120
100
80 Principle #6
60
40
Measures are used to aid
20
Plan
0
the decision making
Actual
process
# of Units Completed
Project Status Date
Project: OLDMS Date Prepared: 1 Oct
2009
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 27F]o] Ç } v
Reporting Results
–Use a systematic analysis process as the basis for
reporting results:
–identify problems Principle #7
–assess problem impact Analysis results are interpreted within
–forecast outcome the context of other information and
–evaluate alternatives data (integrated analysis)
–Example:
Problem Impact Outcome Alternatives
Test planning not Test Plan - Medium Testing will be late or Shift Test Planning to
started Development - Low quality will suffer if Analysts
Development is Staffing - High ovv]vP Test Component C last
slightly behind in the next 2 weeks Extend schedule
schedule with If staffing lag L - (( v}v
Component C continues or worsens, project-related
Staffing is below project will be late commitments
planned levels
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 28F]o] Ç } v
Make Decisions and Take Action
–The measurement results must be clearly Principle #8
understood by the decision makers Measurements provide objective
information to facilitate effective
communication
–Make results available openly, to all those affected
–Action must be taken to realize any benefit from measurement
–Desired actions may not be possible -may have to optimize within existing
constraints
Principle #9
Measurement and Analysis techniques are applied to the
Measurement and Analysis process
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 29AgendaF]o] Ç } v
Measurement and Analysis Process Overview
Plan Measurement
Perform Measurement
Evaluate Measurement
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 30F]o] Ç } v
Evaluating Measurement
Evaluate the
Evaluate the measures
measurement process
Update the experience Identify and implement
base improvements
Like the products and processes it serves, the measurement process periodically
needs review and analysis, too
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 31F]o] Ç } v
Summary
Planning identify key issues and information needs
specify measures that assist decision-making
Measurement
collect measures and verify the data
Performing
analyze the data and generate indicators
Measurement
use the measurement results for decision-making
Evaluating
review the products and process of measurement,
to identify improvements
Measurement
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 32F]o] Ç } v Questions
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 33F]o] Ç } v
PPQC Services
Consulting
CMMI
Software Engineering
Systems Engineering
Process Improvement
Appraising
SCAMPI A, B, C
Gap Analysis
Training
CMMI/Process Improvement
Action Planning Workshops
Measurement and Analysis
Process Area Specific Training
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 34F]o] Ç } v
Contact Information
Address: Henry Schneider
President/Senior Principal Consultant
Process and Product Quality Consulting, LLC
2111 Heather Green
Houston, TX USA 77062
Phone: 281-218-6682
E-mail: henry@ppqc.net
Web Site: www.ppqc.net
8/5/2008 ®2008 ***Proprietary*** All Rights Reserved - Process and Product Quality Consulting, LLC 35
