Sticking Point: Temperature Control Vital to Vaccine Viability


Flu season typically peaks between December and February, but by the time the winter holidays roll around, many of us will have already waited in line at area clinics, grocery stores, and pharmacies to get our annual flu shot. The Centers for Disease Control reports that U.S. vaccination efforts since 1994 have prevented an estimated 16 million illnesses every year.

Even so, some people choose to avoid vaccination, citing reasons like a lack of confidence in its usefulness, complication risks, or religious beliefs. Some news outlets thrive on the controversy, sensationalizing reports of disease outbreaks, possible vaccine failures, and anti-vaccination propaganda. Yet the public debates have missed a critical factor in vaccine effectiveness: temperature.

That’s right. To work correctly, all vaccines require strict temperature control from the point of manufacture up until they are injected into a patient’s arm. Our work at NIST is helping to ensure that providers know how to maintain vaccines at the right temperatures, so those who get the shot can be confident that every dose works as intended.

Most vaccines administered in the United States must be kept between 2 and 8 degrees Celsius through a network called the cold chain. Refrigerated vaccines exposed to temperatures outside this allowable range lose their potency. Vaccines that are too warm will eventually spoil, and a single freezing event will cause irreparable damage, rendering the medication entirely useless. A 2007 meta-analysis estimated that as much as 35 percent of the world’s vaccines are subjected to accidental freezing in the cold chain.

Unfortunately, there’s no quick and easy way to determine whether a vaccine has been exposed to inappropriate storage temperatures, which means ineffective vaccines may be unknowingly administered to patients. Instead, pharmaceutical manufacturers, distributors, and immunization providers must rely on temperature monitoring devices placed in temperature-controlled storage units and shipping containers to assess vaccine viability.

The Centers for Disease Control (CDC) plays a major role in managing the U.S. vaccine market through its Vaccines for Children (VFC) program, which distributes nearly $4 billion worth of vaccines to underinsured and underprivileged children each year. Over 44,000 U.S. clinics distribute vaccines through the VFC program, and immunization providers around the country look to the CDC for guidance and policies on the best methods for storing, transporting, and monitoring vaccines under proper, temperature-controlled conditions.

In 2009, the CDC asked the NIST Thermodynamic Metrology Group for help solving problems in temperature measurement and control of vaccines distributed through its VFC program. Wasted vaccines from equipment and temperature monitoring failures at clinics around the country were adding up to a significant monetary and public health cost. When vaccine temperature excursions are discovered, costly and embarrassing re-vaccination efforts may be needed, which put communities at risk and erode public confidence. Many physicians have expressed frustration at the lack of consistent guidance and information for safely managing their vaccine supplies.

Our group is gradually bridging this knowledge gap through studies designed to replicate the vaccine storage and transport conditions that occur at the provider office level. We study vaccine refrigerators ranging from the cheapest dorm fridge available at your local big box outlet to highly-engineered, purpose-built pharmaceutical and laboratory refrigerator systems.

Each of our test refrigerators is filled with vaccines and wired up with a boatload of miniature temperature sensors called thermocouples. They are then subjected to the sorts of conditions we expect to happen in real life—repeated door opening, temperature adjustments, power failures, overloading the unit with large amounts of vaccines, and so on. We test temperature monitoring devices and setup methods to determine the best way to accurately track stored vaccine temperatures inside a unit without too much effort or expense. And we develop vaccine transport “packout” methods using readily-available materials and coolers.

From our results, we provide the CDC with user-friendly guidance on questions like: What type of vaccine refrigerator should I buy? What should I look for in a temperature monitoring device? How can I calibrate my vaccine refrigerator thermometer? How can I safely transport vaccines in an emergency? How can I best manage power failures?

One of our earliest findings was that dorm-style refrigerators are extremely unsafe for vaccine storage, as they may result in rapid, accidental freezing of vaccine. At the time, these units were commonplace in smaller clinics due to their convenient size and small price tag. Since the NIST findings were published, dorm style units have been removed from all VFC provider offices.

NIST research has generated other major improvements in vaccine management practices, including widespread use of calibrated, traceable, continuously recording temperature monitoring devices inside vaccine refrigerators—phasing out analog devices and handwritten temperature record sheets. Our studies also highlighted the importance of monitoring stored product temperature instead of air temperature, reducing vaccine waste caused by false temperature alarms.

NIST findings are repackaged and distributed through the CDC’s Vaccine Storage and Handling Toolkit, which vaccine providers, state health department immunization managers, and cold chain equipment manufacturers rely on as the primary guidebook for U.S. vaccine management policies. So if you get the flu shot and don’t get the flu this year, give some credit to CDC, NIST and a better-working cold chain.

And the next time you hear people arguing about the flu shot, you can ask what they think about cold chain management practices. It may end the discussion—or prompt a Google search.


About Author

Michal Chojnacky

Michal Chojnacky joined the NIST Thermodynamic Metrology group in 2009. Her work at NIST helps companies and other laboratories measure temperature accurately to regulate manufacturing processes and public utilities, keep products safe, and advance research and development in fields like aerospace, pharmaceuticals, and metrology. She prefers warmer climates to cold.


  1. Dear Ms Chojnacky,

    I am aware of calibration in terms of establishing metrological traceability; however, this argument cannot be used for VVMs since its purpose is different than the contact thermometer. A thermometer is a measuring device for determining the temperature, but VVM is not a temperature measuring device. VVM reacts to the cumulative effects of time and temperature similar to how products like vaccine degrade with time and temperature. VVM develops increasing color with increasing cumulative temperature exposure and provides a simple visual cue to warn health workers to make informed decisions on the extent of temperature exposures and whether to use or not to use the vaccine. Thermometers cannot be compared with VVMs. VVM traceability from the certificate of analysis point of view (as described in the USP ) is defined in WHO/PQS VVM performance specifications WHO/PQS/E006/IN05.2 (19 JAN 2012).

    As for the strict 2-8 deg C temperature regime, there are examples in the U.S. market that products may be exposed to temperatures above 8 deg C for limited periods of time as defined by the manufacturers. For example, Gardasil 9 can be administered provided total (cumulative multiple excursion) time out of refrigeration (at temperatures between 8 and 25 deg C) does not exceed 72 hours.

    Similarly Gardasil Qaudrivalent can also be used out of refrigeration (at temperatures at or below 25 deg C) for a total time of no more than 72 hours (

    Prevnar 13 is allowed to arrive at temperatures between 2 to 25 deg C after shipping (

    Of course, these data are not recommendations for shipping or storage, but may guide decisions for use in case of temporary temperature excursions.

    The most critical point is: Since none of the temperature monitoring devices used along the cold chain are interconnected, exposures to temperature excursions occurring at different legs of the cold chain cannot be mathematically accumulated to judge whether it is still below 72 hours. A vaccine manufacturer making a decision based on the refrigerator temperature measurement provided by the physician assumes that everything was OK prior to the vaccine reaching that particular service point, simply because the data logger used by the physician is not interconnected to other devices used upstream in the cold chain. Regardless of data coming from a calibrated data logger at the physician’s office, it lacks the “full” history of temperature exposures of that particular vial. This approach gives only a “false security.” At this point if VVM was attached it would have the entire history and be the answer.

    The other issue is the belief that science is the same around the world. If vaccines with VVMs are safely used around the world, and these VVMs are affixed on the vaccines by vaccine manufacturers based on the stability profile of the product, why is this science ignored when it comes to U.S.? When there are no clouds during the day, the sky is blue both in Gabon and in the U.S. If we believe in science, why wouldn’t VVM would work in Gabon and also work in the U.S.? Would you agree that VVMs alone would prevent a lot of staff time unnecessarily spent on the phones even when there is half a degree C excursion for some minutes in a refrigerator?

    Last month in New Hampshire it was determined that vaccines were not properly refrigerated for a period of 14 months. The hospital contacted the families of more than more than 800 patients, asking them to bring their children in to get new shots. VVMs used on those vials could have provided an early visual warning that vaccines weren’t being stored correctly and the cost, concern, inconvenience, the loss of confidence in the cold chain at the hospital and the need for revaccination could have been avoided. Thermometers were used in the practice and were likely recording accurate temperatures but not identifying an obvious lingering problem.

    Ed Kostyrna asked “Maybe you could put a ‘spoilage’ indicator on the bottles that would change color permanently if the temperature fell outside the acceptable range.” The answer to Ed is that indicators exist and should be put on each vial of vaccine and other temperature sensitive medicines.

    Warm regards,

    Dr Umit Kartoglu, MD, DPH
    Scientist, World Health Organization
    Department of Essential Medicines and Health Products
    Geneva, Switzerland

    • Michal Chojnacky
      Michal Chojnacky on


      Vaccine manufacturing, packaging, and distribution processes are regulated by the FDA. As a result, widespread use of VVMs in the U.S. would require a joint effort from U.S. drug manufacturers and the FDA. In the U.S., these entities maintain vaccine stability data, and are jointly responsible for ensuring that vaccines are delivered to distribution sites and/or directly to providers under conditions that preserve drug potency.

      Unlike the FDA, both NIST and CDC are non-regulatory agencies. The CDC oversees publicly funded vaccines distributed through the Vaccines for Children (VFC) program, which operates at the provider-office level. NIST provides technical expertise in the fields of measurement science, rigorous traceability, and standards development and use. By identifying and educating providers about easily implemented solutions for better vaccine management–from storage units to temperature monitoring devices, to calibration methods and storage and handling techniques–joint NIST-CDC efforts have prompted dramatic improvements throughout the provider-to-patient segment of the vaccine delivery chain.

      Again, thank you for providing your input and the WHO perspective on the use of VVMs in the global vaccine supply chain. If you’d like to continue this discussion, please feel free to contact me directly.

  2. Dear Ms Chojnacky,

    I find your commentary comparing traditional contact thermometers to vaccine vial monitors (VVM) to be inaccurate and potentially misleading since the purpose of the devices is completely different. VVM is the only temperature indicator that is attached to the product at the unit level and monitors the entire cold chain from the manufacturer until the moment it is used at the service level. No matter how sophisticated, accurate (and calibrated) electronic devices are that are used along the supply chain, they can only be helpful at the particular level and leg of the supply chain where they are used. For example, the World Health Organization also recommends inclusion of electronic shipping indicators in all international shipments, despite the fact that VVMs are on each vial of vaccine. This is due to the fact that in an international shipment, the mission of these electronic devices is over when the consignee receives the package. If there are some deviations during the international shipment and product is accepted based on its “stability budget,” there is no way that this information could be passed on to the lower levels in the system where additional excursions may be possible. When the products move into the central cold storage, they will be monitored through various other electronic temperature devices and systems. None of these devices will have the “memory” of what has happened to the product before it reached this point. This lack of accumulated temperature history perpetuates until the product reaches the practitioner at the service point. Even when the “best” temperature data logger is used at this level, these data cannot be used to determine the entire product temperature history and product efficacy since there is no information on the “actual” remaining stability budget of the product at time of receipt. Data generated using current devices that monitor storage and transport along the supply chain are not interconnected. In contrast, VVM brings a critical and novel approach to this: the health worker, the nurse, and the doctor can simply look at the color of the VVM on a particular unit of vaccine and can tell how the product was handled along the entire supply chain and assist in making an informed decision on product status.

    It was surprising to read your supposition that VVMs cannot be calibrated in the same way that a traditional thermometer is calibrated and with the conclusion that “… their accuracy may be more difficult to pin down.” Though VVMs are not and cannot be calibrated in the same way that a traditional thermometer is calibrated, it is not possible to calibrate any individual single-use device because the test is, by the nature of the time temperature integrators (VVM in our case), necessarily destructive. If you refer to “United States Pharmacopoeia (USP) Monitoring devices – Time, Temperature, and Humidity,” under “Calibration of temperature – and humidity monitoring devices,” it is clearly explained as follows: “Single-use electronic and chemical indicators should follow Good Manufacturing Practices with appropriate testing controls. Electronic indicators require proper calibration. Single-use indicator performance can be qualified by the supply chain user by sampling and testing of multiple production lots. For TTIs that calculate MKT, the performance of a batch can be assessed statistically by subjecting an appropriately sized sample to elevated temperature conditions for a set period of time and observing the results. Manufacturers should adopt appropriate acceptance criteria. It is acceptable to use the release test performed by the manufacturer of the indicator (based on the certificate of calibration or the certificate of analysis and the expiration date) in lieu of calibration or qualification.” Therefore, the statement regarding the calibration of VVM is scientifically inaccurate.

    VVM is a technology, explained in the same USP document, that provides early warning if the product (vaccine) is exposed to excessive heat over time before the expiration date. It is the only temperature monitoring tool that follows the product from manufacture to administration and accumulates the experience across the entire cold chain. VVM has had a profound impact around the globe. Due to the use of VVM, health workers can ensure that vaccines administered have not been damaged by heat. Importantly, VVM also helps to reduce vaccine wastage; it facilitates immunization outreach and increases access and coverage; helps to pinpoint cold chain problems; help to manage vaccine stocks and dispatch; and it prevents inadvertent freezing of vaccines.

    Finally, the technologies WHO recommends are not only for middle and lower income countries; WHO recommends technologies to be used in “programmes” by all Member States. In this sense, if the cold chain rules are global and VVM works, then VVM should also work in the U.S. as well. Given its important advisory role to the U.S. CDC, I suggest that NIST should be open to explore the benefits and proven experience of all temperature monitoring devices including VVM and demonstrate the difference VVM could bring to the U.S. immunization programme.

    Warm regards,

    Dr Umit Kartoglu, MD, DPH
    Scientist, World Health Organization
    Department of Essential Medicines and Health Products
    Geneva, Switzerland

    • Michal Chojnacky
      Michal Chojnacky on

      Hi Umit! Thank you very much for your insight on the role of VVMs throughout the global vaccine supply chain. I wholeheartedly agree – VVMs fill a unique and critical role in vaccine management, by providing a visual indication of cumulative time-temperature history throughout the transport, distribution, and storage of vaccines in the cold chain.

      In terms of calibration, I am aware of lot testing and other sampling methodologies used to quantify VVM performance. These methods are useful for providing an estimate about device accuracy, and as you’ve stated, are compliant with USP requirements. At NIST, we have a slightly different perspective and vocabulary when it comes to “calibration,” since we tend to think about device calibration in terms of establishing metrological traceability ( For a measurement to be traceable, the measurement result must be “related to a reference through a documented unbroken chain of calibrations, each contributing the measurement uncertainty,” according to the International Bureau of Weights and Measures (BIPM) definition. Since it’s not possible to individually calibrate VVM devices (the test would be destructive, as you mention), it seems equally challenging to establish measurement traceability for these devices. By contrast, a contact thermometer with continuous logging capabilities can be calibrated on a periodic basis with documented uncertainty and measurement traceability.

      There’s another issue at play that prevents U.S. immunization programs from adopting certain approaches modeled by the WHO. U.S. vaccine providers are subject to the strict 2 °C to 8 °C storage temperature range specified on the product packaging. If there is any excursion outside this range, even half a degree, providers are instructed to call the vaccine manufacturer for guidance on how to proceed. As you know, many vaccines can tolerate cumulative exposure to temperatures above 8 °C for finite periods of time, depending on the product. VVMs take advantage of this fact, as the selected indicator is designed to approximate the properties of the vaccine inside the vial. However, current U.S. policies and laws don’t really allow our providers to fully utilize vaccine stability data and VVM indicators for every-day decision making about vaccine efficacy. Instead, this authority rests with the pharmaceutical manufacturers, who make their decisions based on refrigerator temperature measurements provided by the physicians. In this scenario, traceable temperature measurements that accurately reflect stored vaccine temperatures play a critical role in preventing unnecessary vaccine waste as well as inadvertent administration of spoiled vaccine.

      VVM technology has revolutionized safe vaccine distribution practices around the globe. In my opinion, VVMs play a critical, supportive role to continuous monitoring systems using calibrated contact thermometer sensors. Together, both products support improved vaccine management throughout the cold chain.

  3. Vaccine failure due to vaccine effectiveness:temperature,is keeping me worrying all night,ineffective vaccine unknowingly administered to patients,a re-vaccination will increase cost and erode public confidence of vaccines effectiveness,patients will be at the receive end.

  4. Sticking Point,temperature control vital to vaccine viability.We patients are worry about unknowingly administered bottle of spoilage vaccines that give false hope to patients.We need some monitoring measure to know the effectiveness of the vaccines,if not,patients will again on the receiving end.

  5. I was going to get a shot at my local VA and asked the tech about how the bottle she was using was maintained at the proper temperature. She could not answer and seemed quite annoyed that I would even ask. I left!
    Maybe you could put a ‘spoilage’ indicator on the bottles that would change color permanently if the temperature fell outside the acceptable range. I big RED dot that could not be removed by the clinic if the bottle ever froze or was exposed to higher than recommended temperatures.

    • Michal Chojnacky
      Michal Chojnacky on

      Hi Ed – a number of commercial solutions, called vaccine vial monitors, do exactly what you’ve described. There are several types of stickers and indicators for use on vaccine vials that will change color in response to an out-of-range temperature condition. “Time temperature indicators” change color in response to cumulative exposure to warm temperatures, and freezing indicators change color in response to a single freezing event. The World Health Organization currently uses these types devices to monitor vaccine deliveries around the world. The biggest challenge with these types of devices is that they can’t be calibrated in the same way that a traditional contact thermometer is calibrated, so their accuracy may be more difficult to pin down. At this time, U.S. immunization policies favor the use of continuous data logging thermometers in the cold chain for the purpose of determining vaccine temperature history and product efficacy.

Leave A Reply