Earned Value Management (EVM) techniques are commonly used as a disciplined process to track Key Performance Indicators (KPIs), Cost and Schedule. Typically for long duration projects which may span several major subprojects, these techniques can be extremely useful to gauge the team’s ability to adhere to a cost budget or a critical timeline for the project. A clinical study is a classic use case for applying EVM techniques as there are major subprojects within the project spanning different resource types. Additionally, the monetary value of clinical studies ranging from ~$5M in a phase 1 trial to exceeding $100M for a Phase 3 trial offer the opportunity to realize substantial financial savings through crisp execution using KPIs as benchmarks through the study.
The fundamental premise for the two KPIs, Cost Performance Indicator (CPI) and Schedule Performance Indicator (SPI) is to incrementally check the progress of a project at set intervals to see if the project is tracking to its baseline. Typically in a project the health check of progress is on a monthly basis, but that frequency can be tailored to best fit the needs of a specific project. The quantitative means to assess how a project is performing can be described with three outcomes:
Cost or Schedule is:
Ahead of Schedule – CPI or SPI > 1.0; below budget
On Schedule - CPI or SPI = 1.0; on budget
Behind Schedule – CPI or SPI < 1.0; exceeds budget
Looking at the KPIs on a static basis is not as informative or helpful but focusing on its trends over each period is where decisions can be made to affect the outcome is a key benefit. Here is a scenario on how this may be helpful:
The site startup subproject is estimated to take 3 months – this is critical to complete in this time frame as it drives enrollment and other downstream subprojects.
2 months after start of the project, no progress was made on site startup, indicative of an SPI of 0.0, however 2 months of labor, as budgeted for that duration, were expended. When presented with this status, an intelligent conversation with the project team can be had with this data to assess how to recover this projected schedule impact and minimize a brewing cost growth concern as well resulting in an unfavorable CPI (no schedule progress, yet expenditures of 2 months). Perhaps for this study completing the study on time takes precedence over the costs incurred and one outcome could be to increase project expenditures, further exacerbating the CPI, but offering a recovery opportunity for SPI.
While looking at the KPI data in the same period as the site startup, a different aspect of the project, perhaps EDC development and IRT setup and integration were tracking for completion ahead of schedule and requiring less cost to complete (favorable SPI and CPI) – this could offset the site startup setback. Using EVM techniques allow a Study Project Manager to understand, quantitatively, the impact one study aspect and take proactive measures to recoup from that setback to minimize missing key project objectives.
If this discipline was applied consistently KPIs could help:
- Study Execution – use of KPIs to steer the ongoing study to reach the desired outcome perhaps at the expense of cost and schedule – these KPIs allow a quantified means to make study decisions.
- CRO selection – perhaps CRO A historically performs well to budget (favorable CPI), but constantly misses deadlines (poor SPI) while CRO B is consistently meeting both SPI and CPI – in a competitive down select process, KPIs can be used to influence vendor selection
- Lessons Learned – Perhaps Database Development consistently overruns their target budget while Clinical Monitoring underruns budget – KPIs can help influence future study funding allocation
- Historical Basis of Estimates: Perhaps when evaluating costs for a new study, having KPIs to substantiate costs can elevate the credibility of the proposed offer.
This approach can also be applied at a program basis in Functional Sourcing Provider (FSP) relationships. Similarly, coupling output productivity of an FSP arrangement in various disciplines and comparing it to the individual project schedules provides informative data on FSP performance. Key actions from these assessments can be to fine tune overall resource allocation, incorporate different types of resources to better align with the project team, enhance underlying processes to better utilize FSP resources, etc.
This blog did not elaborate on the mechanics of setting up a budget and schedule to calculate the KPIs. While this is certainly not a trivial activity, having the project team develop this structure allows for a strong alignment of the overall study is invaluable. Future blogs can elaborate on estimating the time/cost needed for establishing a comprehensive project plan to allow for tracking of these KPIs.