What is the value of variance analysis reporting (VAR) from the perspective of the customer? How will the customer assess the quality of the VAR? This is an important component to consider.
In previous posts, we talked about the significance of and reasons for writing a monthly VAR. We also discussed the components of a quality VAR and considerations for the contractor control account manager (CAM) to contemplate in preparing the VAR.
Those of us who have been in EVMS business for many years know that no matter how hard you try and or how much time you devote to preparing CAMs, VARs are going to take a “hit.” A corrective action request (CAR) or an opportunity for improvement is certain to surface.
The VAR analysis process is somewhat subjective. Not every customer analyst will view the VAR data the same way. However, there are specific areas that the customer analyst will focus on.
Here we explore those areas to provide some guidance on how to successfully produce a quality VAR that is acceptable to the customer.
Most government agencies require their vendors to use the CPR/IPMR Format 5 [refer to DI-MGMT-81861 A] in preparing and submitting their VARs. The major components of the VAR are clearly identified in the IPMR Format 5 – Variance Explanations.
Avoid Common Data Issues
VAR problems generally arise because the data:
- Are not clear or concise
- Are not precise enough to explain the situation to the customer (e.g., root cause analysis, impact statements or corrective action plans)
- Are not recorded in the location as prescribed in the IPMR Format 5 instructions
- Are totally omitted
Attention to detail on the part of the contractor CAM and the project control staff will alleviate many of these problems.
Establish Reasonable Control Account Values
VARs are required any time a cost or schedule variance (current or cumulative) or variance at completion (VAC) exceeds established thresholds. These thresholds are typically established by the contractor in conjunction with the customer.
Since the total dollar value of a project’s control accounts varies, it is best to use both a fixed dollar value and a percentage of the control account value in establishing thresholds, such as +/- 10% AND +/-$200K. The goal should be to explain at least 80% of control account variance dollar value.
In addition, the customer may impose a limit to reportable variances. For example, the top 15, cost, schedule and/or variance-at-completion. This limits the number of VAR submissions to the customer.
Carefully Detail Root Causes
The customer will also typically focus their analysis of the VAR on the contractor’s assessment of the root cause of any variances. In many cases, the contractor will identify a symptom of the variance instead of the root cause. Effective corrective actions cannot be implemented unless the true root cause is correctly identified.
A Six Sigma process utilizing the “5 Whys” technique in conjunction with Ishikawa’s Fishbone Diagram and brain storming techniques typically helps to identify the root cause. This is particularly true in cases where the control account and the issues surrounding the variances are complex.
Outline Impacts and Corrective Action
The customer will also focus on assessing the VAR impact statement. They want to know:
- The cost and schedule impact, positive or negative, that reported variances are having, or will have, on the control account
- Associated control accounts
- The critical path
- The total program estimate at completion (EAC).
All the above should be clearly stated in the VAR Impact Section of Format 5.
As discussed above, identifying the root cause(s) of the control account variances is a critical step in identifying an effective corrective action. The contractor should clearly discuss what needs to be implemented, when the implementation will be completed, and who is responsible for managing the corrective action. Likewise, a discussion of the expected outcome as a result of the corrective action implementation should be provided.
Detail the Estimate at Completion (EAC)
The contractor’s reported EAC will be highly scrutinized. The customer will want to know the reasons for changes in the monthly EAC. This should be explained in the VAR.
Likewise, the customer will be particularly concerned about the realism of the EAC. Are future work scope and associated risks taken into consideration?
Validity checks of the EAC using TCPIeac vs CPIcum should be discussed. This is particularly true if the difference between the two metrics is .10% or greater. In addition, a discussion of the program manager’s EAC compared to the Independent EACs (IEACS’) Best Case, Worst Case, and Most Likely should be provided.
Pull Essential VAR Components Together
Writing an effective and accurate VAR can be a time-consuming challenge. As customer requirements or expectations increase, the challenge to meet VAR submittal requirements will increase as well. The posts in this series should arm you with the essential tools to successfully meet customer expectations.
Winter 2020 IPM Division Meeting Coming in January
/in EventsThe Winter 2020 Integrated Program Management (IPM) Division Meeting will be held January 29 – 30, 2020, at L3Harris Technology Center in Palm Bay, Florida.
Come to the NDIA Integrated Program Management Division Meeting to exchange views and information to promote and communicate integrated program management best practices. Each meeting offers a different look into the role of integrated program management with regards to the defense and security needs of the nation.
Visit the event page for details.
IPMW 2019 to Focus on Project Performance Techniques
/in EventsIPM Workshop 2019 is your chance to learn about project performance techniques, including earned value management (EVM). The workshop provides training, current topics from practicing professionals, access to world leaders in project performance management, networking and the latest in software and consulting to make your program successful.
This event is a great way to learn the latest project performance techniques.
The event will take place at the Hyatt Regency Baltimore Inner Harbor. Details and registration information are available.
Earned Value Management Practitioners’ Forum Coming in October
/in EventsThe 9th Annual Earned Value Management Practitioners’ Forum will be held October 9-10 at the UVA Darden Conference Center in Arlington, Virginia.
This conference is for EVM application users to share best practices and help others become more effective in their use of EVM tools, analysis and reporting.
Planned topics are:
Details and registration information are available.
Empower Users’ Group Workshop, Aug 28-29
/in EventsEncore Analytics is presenting an Empower Users’ Group (EUG) Workshop, Aug 28 & 29 in Miramar, Florida.
Speakers from government organizations and industry will talk about the use of Empower for program analytics, data-driven audit support, VAR collection, and Corrective Action Tracking. There will also be a number of technical sessions hosted by the Encore Analytics staff to cover advanced topics.
There will be an open feedback session with customers to prioritize future software enhancements. Attendees will have the opportunity to influence the future direction of the software during this session.
Registration for the workshop is still open.
Meeting Customer Expectations with VAR – Part 3
/in EVMSWhat is the value of variance analysis reporting (VAR) from the perspective of the customer? How will the customer assess the quality of the VAR? This is an important component to consider.
In previous posts, we talked about the significance of and reasons for writing a monthly VAR. We also discussed the components of a quality VAR and considerations for the contractor control account manager (CAM) to contemplate in preparing the VAR.
Those of us who have been in EVMS business for many years know that no matter how hard you try and or how much time you devote to preparing CAMs, VARs are going to take a “hit.” A corrective action request (CAR) or an opportunity for improvement is certain to surface.
The VAR analysis process is somewhat subjective. Not every customer analyst will view the VAR data the same way. However, there are specific areas that the customer analyst will focus on.
Here we explore those areas to provide some guidance on how to successfully produce a quality VAR that is acceptable to the customer.
Most government agencies require their vendors to use the CPR/IPMR Format 5 [refer to DI-MGMT-81861 A] in preparing and submitting their VARs. The major components of the VAR are clearly identified in the IPMR Format 5 – Variance Explanations.
Avoid Common Data Issues
VAR problems generally arise because the data:
Attention to detail on the part of the contractor CAM and the project control staff will alleviate many of these problems.
Establish Reasonable Control Account Values
VARs are required any time a cost or schedule variance (current or cumulative) or variance at completion (VAC) exceeds established thresholds. These thresholds are typically established by the contractor in conjunction with the customer.
Since the total dollar value of a project’s control accounts varies, it is best to use both a fixed dollar value and a percentage of the control account value in establishing thresholds, such as +/- 10% AND +/-$200K. The goal should be to explain at least 80% of control account variance dollar value.
In addition, the customer may impose a limit to reportable variances. For example, the top 15, cost, schedule and/or variance-at-completion. This limits the number of VAR submissions to the customer.
Carefully Detail Root Causes
The customer will also typically focus their analysis of the VAR on the contractor’s assessment of the root cause of any variances. In many cases, the contractor will identify a symptom of the variance instead of the root cause. Effective corrective actions cannot be implemented unless the true root cause is correctly identified.
A Six Sigma process utilizing the “5 Whys” technique in conjunction with Ishikawa’s Fishbone Diagram and brain storming techniques typically helps to identify the root cause. This is particularly true in cases where the control account and the issues surrounding the variances are complex.
Outline Impacts and Corrective Action
The customer will also focus on assessing the VAR impact statement. They want to know:
All the above should be clearly stated in the VAR Impact Section of Format 5.
As discussed above, identifying the root cause(s) of the control account variances is a critical step in identifying an effective corrective action. The contractor should clearly discuss what needs to be implemented, when the implementation will be completed, and who is responsible for managing the corrective action. Likewise, a discussion of the expected outcome as a result of the corrective action implementation should be provided.
Detail the Estimate at Completion (EAC)
The contractor’s reported EAC will be highly scrutinized. The customer will want to know the reasons for changes in the monthly EAC. This should be explained in the VAR.
Likewise, the customer will be particularly concerned about the realism of the EAC. Are future work scope and associated risks taken into consideration?
Validity checks of the EAC using TCPIeac vs CPIcum should be discussed. This is particularly true if the difference between the two metrics is .10% or greater. In addition, a discussion of the program manager’s EAC compared to the Independent EACs (IEACS’) Best Case, Worst Case, and Most Likely should be provided.
Pull Essential VAR Components Together
Writing an effective and accurate VAR can be a time-consuming challenge. As customer requirements or expectations increase, the challenge to meet VAR submittal requirements will increase as well. The posts in this series should arm you with the essential tools to successfully meet customer expectations.
Writing an Effective Variance Analysis Report (VAR) – Part 2
/in EVMSLast month we provided an overview of variance analysis as well as a discussion of items to consider when preparing to write the monthly variance analysis report (VAR). This month we expand on the topic and look at the elements of an effective variance analysis report.
It is imperative that the Control Account Manager (CAM) write a clear, precise, and accurate VAR to inform internal management of issues affecting project performance.
Keys to an Effective Variance Analysis Report
A well-written VAR provides this critical information for external customers to understand the current status and future projections of the project. The goal is to write a quality VAR, one that clearly explains the issues that are causing schedule and cost variances and what steps are being taken to correct those issues.
A VAR is required when the established thresholds for deviations from cost, schedule and at-completion objectives are exceeded. However, since the VAR is an excellent communication tool, the project manager may want to consider also having VARs developed for account variances just below these thresholds.
An analysis of the trends, especially those that are negative, may indicate the potential for future problems. This proactive “getting ahead of the game” approach will save time and avoid playing catch up once an issue arises.
When developing the monthly VAR, both the current period data as well as the cumulative data (i.e., since contract start) will be reviewed. Typically, the current data will show greater fluctuations from period to period than the cumulative data. This is normal. The cumulative data tends to smooth out these large fluctuations over time and thus presents a more useful trend line.
Cost, schedule and at-completion variances, assuming they exceed the established thresholds, are addressed each month for both current and cumulative data. VAR thresholds vary from project to project, are developed in conjunction with the customer, and are expressed in a fixed dollar amount and/or a percentage of the control account value.
A typical VAR form (CPR/IMPR Format 5) clearly identifies these three separate areas. (Refer to DI-MGMT-81861 A) On most projects, the contractor will use a VAR checklist to provide guidance to the CAM as to what data to address.
Major components of the VAR (Format 5)
The State of the EVMS Standard
/in EVMSEarned Value Management requirements in the federal space are typically imposed on contractors via either FAR 52.234-4 Earned Value Management System or, for defense contracts, DFARS 252.234-7002 Earned Value Management System (EVMS).
The two clauses are slightly different. The FAR directs contractors to use an EVM system compliant with the guidelines in the EIA-748 standard. The DFARS clause mandates the use of an EVMS compliant with ANSI/EIA-748. Why the difference, and does this mean that there are two different standards?
A Little History
Throughout this history, NDIA has continued to be the author and steward of the 748 standard.
Today the EIA-748 standard is maintained by the SAE G-47 Systems Engineering Committee and was most recently revised on January 8, 2019 to EIA748D. The rationale for the change includes the following note:
“NDIA Integrated Program Management Division (IPMD) is the author of and the steward for the EIA 748 Standard for Earned Value Management Systems.
IPMD has a large active base of EVM system users and maintains ongoing contacts with the US Department of Defense and other federal agencies regarding the implementation and use of EVMS. More information can be found at https://www.ndia.org/divisions/ipmd.
Is There an ANSI Standard?
Currently, ANSI has no definitive standard for EVM. It does, however, have PMI FS-PMBOK-2017 A Guide to the Project Management Body of Knowledge (PMBOK Guide), which includes ANSI/PMI 99-001-2017 The Standard for Project Management.
The Project Management Institute (PM) has also published a PMI PSF-EVM-2011 Practice Standard for Earned Value Management – Second Edition. This practice standard is not an ANSI standard. As stated in the preface it is designed “…to expand upon the concept of EVM as presented in the PMBOK Guide in a way that allows for easy understanding and implementation.&rdquo
The Project Management Institute (PM) is an ANSI accredited Standards Developing Organization in the area of project management. PMI is currently in the process of developing a Standard for Earned Value Management, which when approved would become a new ANSI Standard.
What About ISO?
Funny you should ask! The International Organization for Standardization (ISO) recently published ISO 21508:2018 Earned value management in project and programme management.
The ISO standard describes 11 process steps required to implement EVM and is designed to complement ISO 21500: Guidance on project management and ISO 21503, Project, programme and portfolio management — Guidance on programme management.
Where Does That Leave Us?
There are two active EVMS standards today which could be relevant to practitioners in the United States: EIA748D, and ISO 21508:2018.
ANSI, through PMI, is in the process of developing a new Standard for Earned Value Management. The public comment period for this Standard should open during 2019.
DOE Project Management News, Feb. 2019
/in EVMSThe February edition of the DOE Project Management Newsletter is shared here as a must read for anyone involved in an Earned Value Management System (EVMS) project or planning to start one.
This issue continues a series on GAO Scheduling Best Practices with the 7th article in the series, this one on “float.” A second article on GAO Cost Estimating Best Practices is also presented. Also get dates for upcoming workshops and training.
NDIA Integrated Program Management Division Meeting Update
/in EventsRooms are now available for the 2019 Spring Integrated Program Management Division Meeting. To secure our group rate at the Hyatt Regency Dulles in Dulles, VA, you must make a reservation before Friday, April 5, 2019. Book online or call the hotel directly at 703-713-1234 and request the NDIA IPMD rate.
The NDIA Integrated Program Management Division Meeting, held April 30 – May 1, 2019, is THE forum for exchanging views and information to promote and communicate integrated program management best practices. it is a great opportunity to interact with government policy makers, broaden your professional network, and contribute to industry guides, white papers, and working groups.
Integrated Program Management Division Meeting Coming April 30
/in EVMSSave the Date! The 2019 Spring Integrated Program Management Division Meeting will take place on April 30 – May 1, 2019 at the Hyatt Regency Dulles in Dulles Virginia.
The NDIA Integrated Program Management Division Meeting is a forum for exchanging views and information to promote and communicate integrated program management best practices.
Visit the event page for details. Registration and the agenda will be available soon.