Instrument Quality Control For Reproducible Flow Cytometry Experiments

The NIH has released a series of reproducibility guidelines that scientists must address. These guidelines have been introduced because there’s a lot of data showing that reproducibility in science is frustratingly lacking in certain experiments.

Non-reproducible experiments waste time, money, and resources. As Begley and Ioannidis cite in their 2015 article :

“The estimates for irreproducibility based on these empirical observations range from 75% to 90%. These estimates fit remarkably well with estimates of 85% for the proportion of biomedical research that is wasted at-large.”

Reproducibility is a mindset, and involves an overall analysis of the scientific process to identify the areas that can be improved. In this article by Bruce Booth, he reviews “Begley’s six rules”. Two of these rules focus on the controls and reagents used in the experiment.

Of equal importance are the instruments used to make measurements. For example, how often are the pipettes calibrated? Are all lab members adequately trained in technique? This chart from Gilson is a useful one to have in the lab as a reminder on proper pipetting form.

The flow cytometer is an integral component of any flow cytometry experiment, and special attention should be paid to ensuring that it is working correctly and consistently. As an end-user, the researcher should be able to sit down at a machine and know that it is performing the same way today as it was yesterday and last week.

Equally important is that if any changes in instrument performance have occured, the end-user knows how they have been addressed and corrected, rather than letting them fester and potentially affect the results.

For those using core facility equipment, it is important to talk to the people who maintain your instruments and look at the quality control data.

Ask them how are they are assessing and maintaining the quality of the instrument and ask what is the best way for you to make sure your data is consistent. The staff will be delighted to to advise! Quality control measurements can include a variety of targets, such as PMT sensitivity, laser alignment, fluidic stability, background issues, and more.

1. PMT Voltage Optimization.

With the analog flow cytometers, researchers were taught to run an unstained cell sample and adjust voltages so that the background was in the first log decade. This outdated practice continues to be taught, even on new digital instruments.

A better approach is to have optimized PMT voltage settings, and to use these voltages for experiments.

In 2006, Maecker and Trotter published a paper on how to determine the optimal voltage for a PMT. In this case, the second peak of the Spherotech 8-peak Rainbow Calibration Particles (RCP-3-5A-2) is run over a voltage range for each PMT. The robust coefficient of variance (rCV) is calculated for each PMT at each voltage, and a curve is plotted, as shown in the figure below.

Figure 1: Peak-2 characterization of PMTs.

The resulting curve shows a steep descent where the rCV decreases as voltage increases. Eventually, an inflection point is reached where the slope of the line changes. At this point, increasing voltage doesn’t improve the rCV.

The optimal voltage range is defined as the region just after the inflection point. In the case of Figure 1, this is between 400 to 450 volts.

When running a one-off experiment, set the voltage for the detectors to these voltages and begin collecting data. There are very few reasons to deviate from this starting voltage.

The most common situation is if the fluorescent signal on your target cells is out of the linear range of the detector, or off-scale. In this case, one might consider reducing the PMT voltage, but a better approach would be to reduce the fluorescence intensity by staining with a mix of labeled and unlabeled antibody.

When doing this, you keep the total antibody concentration constant, but now the cells will have a lower total fluorescence due to the presence of unlabeled antibody.

However, peak-2 beads are not cells, and don’t necessarily have the same spectral characteristics as the fluorochromes used in the experiment.

If the panel is going to be used consistently over a long period of time, the optimal voltage can be refined by performing a voltage titration (voltration) experiment using stained cells.

An example of this secondary optimization is shown in Figure 2. On the left, is the optimization for CY7-APC and on the right, brilliant violet 650. The peak-2 value for each of these fluorochromes is pointed out with the blue arrow. However, this peak-2 value is not always the optimal voltage for a specific fluorochrome.

Figure 2: Voltage optimization of 2 fluorochromes.

To determine if the peak-2 value is optimal, increase the voltage past this point and calculate the staining index. In the case of CY7-APC, as voltage is increased, the signal very rapidly moved out of the linear dynamic range, as shown by the red arrow, and eventually moved completely off scale. This indicates that the peak-2 value is best for this fluorochrome.

Brilliant Violet 650, on the other hand, shows an increase in the staining index above the peak-2 value. At the optimal voltage for this fluorochrome-detector pair, the SI is about 16% higher than at the peak-2 value, suggesting that this higher voltage is better for making a more sensitive measurement.

Once this voltration is performed, and the optimal voltage identified, it becomes important to maintain performance consistency over time. To do this, another bead is used. In this case, the sixth peak of the 8-peak Rainbow Calibration Particles, or any other bright bead of your choice.

Run the bead at the optimal voltages and determine the target values in each detector. This is typically measured as the median fluorescent intensity (MFI). Each time the experiment is to be run, the calibration bead is run first and detector voltages are adjusted to ensure that the MFI is maintained, within some tolerance range.

Once that is done, begin collecting the rest of the tubes. If the voltage changes are significant, it is good to pause and consult the daily QC, or your core staff, to determine if there has been a change to the system.

Defining “significant” is up the investigator, but a good rule of thumb would be that deviation of more than 10% in voltage should require some investigation. You should collect and record at least 10,000 single beads for your QC tracking.

It is important to remember that Quality Control like this is not valuable unless it is monitored. To do that, use the Levey-Jennings plot. This is a quality control plot that shows the running average of the data with control lines. In the case below, the dotted lines represent +/- 1 and 2 standard deviations around the mean.

This plot can be used to spot trends in the data. For example, if a data point falls outside of the control lines, it is a sign there may be an issue with the system, and intervention may be required.

This could just be cleaning, but may also indicate something else that the manager of the instrument needs to know. If, over time, there is a trend towards one direction or the other (continued increase and/or decrease), it is also a good time to intervene, if for no other reason than to see if other issues with the system have occured.

Figure 3: Levey-Jennings plot over time tracking PMT voltage changes using peak-6 bead method.

As shown in Figure 3, the data on this instrument over 100 days within 2 standard deviations of the mean, giving added confidence that the instrument is working consistently and changes in the data are more likely due to biology, and not instrument issues.

If these start to fall out of this range, then you would need to investigate and troubleshoot. You can be more restrictive and use only one standard deviation, but the industry standard is to use two.

While instrument QC is usually left to the manager of the instrument, adding experimental QC using an independent measurement will aid in improving the quality, and ultimately reproducibility, of the data being generated. Something may change on the instrument between when the manager checks in the morning and when you sit down to run, so it’s wise to check performance for yourself.

2. Parallel Arrangement and Fluidics Stability.

Most of the current instruments have a parallel laser arrangement. This means that the cell passes through lasers in sequence, so they are exposed to only a single excitation source at a time. To ensure that the correct signal pulses are paired together, a factor called the “time delay” is calculated during QC.

If the time delay is off, this will adversely impact the data as the cells arrive either too early or too late, thus the correct pluses are not paired together.

This can easily be identified retrospectively, but it is better to identify the issues during acquisition. If there is a loss of signal on a given excitation source, it can suggest a clog in the system that needs to be cleaned.

Consider a partial clog on the input side of the cytometer (before the flow cell). When the diameter of a pipe is reduced, that increases the flow (speed) of the liquid in the reduced diameter. This results in cells arriving at the interrogation point more quickly than the time delay is expecting them.

A clog after the flow cell has the same issue, but now the pressure builds up behind the clog, slowing down the stream, and cells arrive after the time delay window. This can be seen in Figure 4.

Figure 4: Impact of clogs on data.

When running your experiments, put up a plot of a bright fluorochrome versus time for each of the lasers that are being used on the machine. This will help determine if there are problems with the fluidics, as well as with the lasers.

In the example shown here, PE-Cy5.5 was being excited by a green laser which was the fourth in line, but there was a clog causing a back pressure issue. The clog caused a speedup of the number of events because it constricted the flow core and increased the flow rate in the number of events.

Eventually, the clog worked itself out and the signal came back. As with any quality control metric used, it’s important to not only run theses controls, but to look at them on a regular basis.

3. Compensation.

Compensation is one of most important steps in the flow cytometry process. For proper compensation it is critical to bring the right controls to the instrument every time. This is such an important topic, it has been discussed extensively here. It is here in discussing instrument reproducibility as compensation controls also give insight into how the system is working, and provide a secondary proxy for instrument QC.

These compensation controls must follow the 3 commandments:

1. Controls need to be at least as bright as any sample to which you’ll apply to your compensation. That means, the control signal needs to be at least as bright as anything in your experimental signal. If it’s not as bright, then your compensation will be incorrect. It also has to be within the linear range of the detector.

2. The background fluorescence should be the same for the positive and negative control populations for any given parameter.

Compensation is a property of the fluorochrome, not a property of the antibody or a property of the carrier that brings the fluorochrome to the intercept point. This is why you can use either beads or cells. What’s important is that the background fluorescence, the autofluorescence of the negative population and a positive are the same. Since compensation corrects to the background of the control, it is critical to have a positive and negative in each control. This means it is possible to mix beads and cells in a compensation matrix as long as each positive is linked to the appropriate negative particle type.

3. The compensation colors must be exactly matched to your experimental color.
Alexa 488 cannot be used to compensate for GFP or for FITC. Likewise, if there are two different lots of tandem dye, make sure that lots are matched in the experiment and control. One lot of PE-Cy5 cannot be used to compensate a different lot of PE-Cy5 and vice versa, because each of these dyes are manufactured, and likely have different spectral characteristics.

BONUS 4. You need to have enough events!

This is a minimum of 10,000 single events, when using beads. When using cells, you need a minimum of 30,000 single cell events. Collecting more is good, but don’t collect less than these minimums.

Since fluorescence is a property of the fluorochrome, not the carrier, the question arises, “Should you use beads or should you use cells?” Either carrier is fine, as long as the positive populations have the same background fluorescence when unstained — i.e. if you use beads, then make sure you have unstained beads and if you use cells, make sure you have unstained cells. If you have some on beads and some on cells, then make sure you pair each positive with an appropriate negative.

4. Quality Control.

Often, the QC of an instrument is left to the manager of the instrument. This can be done in a variety of ways, from an instrument-specific QC protocol or something from the manufacturer, like the BD CS&T or the Attune performance tracking beads.

This process should be run every time the instrument is turned on, to ensure that a library of instrument behaviour is developed and to allow the users to understand how the system is performing. QC of this nature is critical best practices for the instrument, and provides baseline confirmation the system is working adequately.

Don’t hesitate to ask the manager of the instrument to look at their QC metrics, and ask for an explanation when things don’t seem clear. After all, the success or failure of the experiment lies in the hands of how well the flow cytometer is working, and most cytometrists will be overjoyed to talk shop!

In conclusion, one key area to improve consistency and reproducibility is to monitor instrument quality control. Knowing how the system is QCed and what happens when something goes wrong is paramount to ensuring that the instrument is properly characterized.

To learn more about Instrument Quality Control For Reproducible Flow Cytometry Experiments, and to get access to all of our advanced materials including 20 training videos, presentations, workbooks, and private group membership, get on the Flow Cytometry Mastery Class wait list.