The purpose of logistics in humanitarian organisations is to make people, processes and systems work together to support the efficient and effective delivery of goods and services.
Monitoring and evaluation (M&E) are integral parts of the logistics management process and provide a link between planning and implementation. While monitoring focuses on the activities organisational logistics perform and their outputs, evaluation focuses on the outcome and goals achievement.
Monitoring is the continuous process of gathering logistics and programme information to measure against previous base-line indicators that are aligned to the goals and objectives of a program. A continuous review of the degree to which a logistics activity is completed and if its objectives are being met allows for corrective actions to be taken.
Evaluation is the continuous measurement process of the quality of the output a logistics function or service provides to analyse progress towards meeting established objectives and goals. Evaluation should be undertaken in such a way that shortcomings can be identified and corrected. It is done on an ad hoc, monthly, quarterly or yearly basis.
Evaluation should also continuously feed into the planning process so that the planned method of intervention can be modified to adapt to the realities and conditions on the ground. Evaluation provides feedback on whether plans have been met and the reasons for success or failure, providing a tool for management to ensure that focus is maintained.
M&E has several purposes:
For the purpose of this document, a user is the final recipient or the identified final beneficiary of the logistics activities and services. There are two sets of users:
Logistics includes a wide-reaching set of concepts that can encompass several aspects, from procurement to distribution, the entire supply chain or the fleet, energy or premises management can fall under the logistics responsibilities.
Each section of this guide contains information on how to monitor and ensure the quality of the service provided by each respective function of logistics. Each individual tool provides information on specific parts of the process, while a wider and holistic monitoring system should be adopted to form a complete picture of the logistics performance as a whole.
Monitoring and Evaluation Tools in this Guide:
|Vehicle and Fleet Management||Fleet Performance Monitoring|
|Inventory Planning and Management|
|Warehousing and Physical Stock Management|
|Electrical Power Generation||Generator and Solar Electric Monitoring|
|Cold Chain||Cold Chain Monitoring|
If efforts are made to stablish a monitoring system, it is better to ensure that the M&E process or activity is important to the continuity of the logistics activities and will have an impact on overall performance. A thorough analysis of the context, objectives, desired outcomes, and organization goals will help to properly define the specific aspects to monitor in depth. The following are some of the aspects that can be monitored in a logistics activity.
Lead time is the time between placing an order and receiving the goods or service. In disaster/emergency relief situations, timing of delivery can have a serious impact on the relief operation and on the beneficiaries.
The internal performance of a logistics function is dependent on the efficiency and effectiveness of each of the individual logistics components. For example, one performance indicator for procurement might be the ability to disseminate information on the number of orders issued. Knowledge of pending orders will allow the warehouse to plan for storage space, while unexpected deliveries can disrupt operations.
The measurement of efficiency is sometimes relative and dependant on what an entity defines as efficiency. In logistics management, efficiency is the satisfactory delivery of a logistics service that enables the end user to fulfil the intended purpose of the request. A good example is the request for medication to be pre-positioned before a malaria season. A late delivery would mean higher incidents of malaria and an increase in the request for malaria treatment rather than malaria prevention drug.
The concept of “total cost” focuses on reducing the total cost of logistics rather than the cost of each activity. An organisation should monitor cost reduction across the board and evaluate the impact on each of the logistics components. For example, purchasing in bulk may reduce the cost of the product but at the same time increase the stock holding costs.
Inventory carrying costs include:
In recent years the concept of value has become accepted as the difference between the value a customer attributes to a product or service and the cost of acquiring the item. Excessive stock holding is not only a risk in emergencies - in the event of an evacuation stocks may be abandoned - but also not cost effective when money is tied up in dormant stocks that may not all be utilised within reasonable time, or used at due to rapidly changing needs. Monitoring and collaborating closely with programs on distribution rates helps in balancing the benefits. Storekeepers are encouraged to share monthly stock reports with stakeholders so they may know what they have in their possession.
Order management costs include those costs incurred for issuing and closing orders, the related handling costs, and the associated communications costs. In other words – the staff and infrastructure costs associated with placing orders, and not just the costs of the items themselves. How many cumulative staff hours does it take to complete a single order, multiplied by their hourly salary? What about the costs of maintaining communications systems and renting office space? It is advisable to benchmark these and keep them under close monitoring to ensure that service delivery is cost effective.
The cost of waste covers the cost of disposing of item packaging, disposing of spoiled, expired, recalled or damaged relief items, or of disposing of damaged, unserviceable equipment. Waste disposal costs have sharply increased due to environmental impacts and national regulations. An overview of environmental cost can be seen in the Green Logistics section of this guide, while information on disposal and national regulation can be seen in the Warehousing Section.
A useful way to measure performance is through the establishment of indicators for the key aspects of logistics activity, with the goal of evaluating the success of an ongoing process or particular activity.
Key Performance Indicators (KPIs) are quantifiable measurements of performance for key activities managed by organization or team. These include all the activities that are needed to keep an operation functioning on an ongoing basis.
A KPI consists of the following elements:
All the information needed to understand the KPI units of measure need to be clear for everyone involved, and when two or more variables are measured, this needs to be clearly defined.
By setting the KPIs, the key areas of the intervention are defined, using a predefined metric (as well as a target value) to indicate how that key area is performing. Thus, KPIs are used to measure the health of an organization and its respective teams and departments. KPIs are often thought of as “health metrics” because they give the vital signs and provide warning signs when the metrics are unusual.
While suitable performance measures provide the foundation for informed decision-making, improper ones can distort the conclusions and negatively impact efficiency by disguising critical issues and warning signs. Good metrics have several distinguishing characteristics:
One of the most complete indicators is the percentage of orders delivered in full, on time and error free (DIFOT).
Orders received on or before the date requested
Orders are complete in quantity
Orders are complete with proper documentation, labelling, and without damage to items or packaging
However, there are a variety of other key indicator examples. Their selection will depend on the specific monitoring needs an organization might have. The following list is non-exhaustive:
Controls are normally put in place to monitor weaknesses, poor designs in projects and improper implementation of programs. Based on the evaluation of outcomes, these weaknesses or shortfalls against targets or objectives set can be corrected or revised in order to continually improve performance.
Each evaluation must be carried out against a pre-established goal that defines not only what is the desire outcome of an intervention, but also the process and needs to achieve it. An evaluation exercise consists of defining the degree of achievement and examining how well or poorly the activities performed have led to those results.
To ensure planed objectives are accomplished, a logistics unit or team should come up with a strategy that will address challenges and guide the teams towards their goals. Organizations should always seek to optimize use or resources to ensure efficient implementation of activities.
Based on analysis and aligned with the general goals of the project, a logistics team or unit should stablish its own ultimate goal or goals that will prioritize operational tasks.
Objectives and key results function as a “road map” to guide the teams to a defined goal. Objectives should be formulated as the desired concrete outcomes, expressed as a positive change expected to be achieved after a defined period and in response to identified challenges. The objectives are reached through the combination of the results that are the effects of the activities.
An objective usually has two to three key results for the same reasons that a GPS device needs two to three satellites to accurately pinpoint a location. Each key result is designed to positively impact a certain metric, remove ambiguity by clarifying and quantifying what success for any given objective looks like, and help measure progress towards that objective.
The composition of a key result looks similar to a KPI, except that a key result includes a timeframe as a starting and target point.
A key result consists of the following components:
Results are the outcomes of different activities measured together as a whole that lead to the achievement of an overall objective. On a day-to-day basis, these activities are the most basic steps to focus on. If well designed, performing every activity will lead to achieving a goal.
To properly define each of these steps it is necessary to create a clear action plan. An action plan will establish a time frame, indicators, persons responsible, and costs of each activity, and should be shared with all persons involved.
A report is used to analyse facts and information to inform the steps towards reaching an objective and possible problems faced, while an evaluation will take this data and establish the degree of achievement and evaluate how a defined strategy and/or plan have worked.
It is important to create a reporting system that will follow the progress of strategic plans, and give feedback on activities of a specific location over a specific time frame. Reports in general should be as concise as possible whilst ensuring all important information is recorded.
The objectives of a report are:
The better the report structure, the easier and more accurate a performance evaluation will be.
An inherent part of monitoring and evaluation relies on data collection. However, a good collection of data does not guarantee the goals and objectives will be achieved. Frequently collected data is only used to respond to and understand past events, instead of being used to drive future actions. To make the most of the monitoring and evaluation efforts, a proper performance monitoring plan needs to be in place for short-term, mid-term, and long-term activities.
Having a measurement process in place ensures frequent, constructive reviews of defined metrics, and creates a culture of measurement and improvement. Staff should be able to see how their performance affects the achievement of the overall goals.
Tracking the progress of KPIs over a period of time means an organization and its teams have clear visibility on the priorities of the organization or project, and enables team members to easily identify trends, strengths, weaknesses, and opportunities. Having this information at hand gives the planners the opportunity to make better, calculated decisions.
KPIs should be carefully picked by closely reflecting on the organization’s strategy and priorities. KPIs transparently communicate what is expected, what should be kept top of mind and how they should carry out their day-to-day activities.
Total number of Framework/Long Term Agreements
% Of projects with Procurement Plans done
Total Number of staff with responsibility to sign a request
Average monthly forecast of logistics expenses
% Of In kind Donation with a donation reference
Monthly total number of Request
% Of Requests presented in ad-hoc regular meetings.
% Of Requests correctly filled and with enough technical specifications
% Of Direct Purchase processes compliant and correctly archived
% Of Negotiated Processes compliant and correctly archived
Average needed time to process and complete a tender
% Of orders fulfilled on time
Total monthly expenditure
Transport and Deliveries
% On time delivery
% Items damaged in transit
% Items lost in transit
Average cost per kg/m3
Average cost per kilometre
Total cargo transported in time frame (kg/m3)
Average delivery time in days
% Of accurate deliveries (sent/received)
% Of movements without Delivery and Reception Notes.
Cost per m2 of covered storage space
% Of stock lost due to theft, spoilage or damage
Number of stock outs per month
|Average % of floor m2 space used per month / Average m3 used per month|
Average time to release stock after pick order received
Number of pest controls per month
Average temperature / Average humidity
Number of temperature alerts
% Of unused stock (out of contingency or more than two years/projects old)
% Of products not damaged or not compliant with specification upon receipt at facility
Total number of vehicles
% Of movement planned in advance
Total number of driver trainings
% Of operating hours vehicles are fully booked
% Of vehicles with the necessary tools
% Of vehicle logbooks filled correctly
Total number of maintenances per vehicle in a month
Average fuel consumption (ltr/km) per vehicle per month
% Of transport request met vs needed
% Of vehicles that meet the mechanical and safety standards
% Of fuel and rental/owned expensed budgeted
% Of equipment correctly codified and labelled
Total number of equipment items
% Of equipment in use
Number of old/obsolete/broke equipment dispose following the organization policy
Information and Communication Technology
Average backups done per month
Total communication cost per month
% Of movements without coverage during a part of the trip
% Of computers with official software license
Total power needed
Average hours without power
Number of power backup system in place
Average power back up maintenance per month
Total electricity cost
% Of equipment powered with stable voltage
% Of installation properly earthed
A logistics evaluation tool is used to monitor the quality of the logistics performance by assessing the level of compliance with organizational guidelines and procedures. The main purpose is to clearly understand where the logistics unit or team globally stands in relation to the application of the internal logistic procedures. Indicators are not necessarily linked to the project specific logistics strategy, but to quantified by internal organization procedures. The purpose of running an evaluation is to have a “picture” of currently operational capacity.
Such a tool can also be used internally to serve as a self-evaluation, looking for areas of improvement or failures of compliance. Ideally, these reviews should be performed by an external person/body not directly involved in the logistics management of that specific location.
USAID, (2006). Monitoring and Evaluation Indicators for Assessing Logistics Systems Performance.
Davidson, Anne Leslie, (2006). Key Performance Indicator in the humanitarian Logistics