As cell and gene therapies (CGTs) move from experimental therapies to real-world treatments, developers face one major challenge:

How do you build a robust, reproducible manufacturing process around something as complex as living cells or gene vectors?

The answer, at least the one industry has tested, lies in Quality by Design (QbD). Before the introduction of Quality by Design (QbD), pharmaceutical development largely followed a Quality by Testing (QbT) model. In this reactive approach, product quality was verified by testing the final product, often without a deep understanding of how process variables influenced outcomes. Manufacturing changes or scale-up often introduced risk, and regulatory filings lacked the transparency to explain why processes worked—only that they did.

To modernize this model, regulatory agencies like the FDA and ICH (International Council for Harmonisation) introduced QbD in the early 2000s through ICH guidelines Q8 (Pharmaceutical Development), Q9 (Quality Risk Management), and Q10 (Pharmaceutical Quality System). The goal: move toward a proactive, science- and risk-based framework where quality is built into every step of product development and manufacturing.

QbD starts by defining the Quality Target Product Profile (QTPP)—the ideal safety, efficacy, and performance profile of the final product. Developers then identify Critical Quality Attributes (CQAs), followed by the Critical Material Attributes (CMAs) and Critical Process Parameters (CPPs) that influence them. Using experimental tools like Design of Experiments (DoE) and statistical analysis, a design space is established to define acceptable process ranges. The final output is a control strategy ensuring consistent product quality through ongoing monitoring.

Table below summarizes definitions, examples and a few references.

Term	Definition	Cell & Gene Therapy Example	Reference
QTPP (Quality Target Product Profile)	The intended quality profile of the product—describes the ideal outcome.	• ≥80% CAR-T potency • Viability ≥70% after thaw • Transduction efficiency ≥30% • Cryo shelf life ≥12 months	FDA Guidance on Human Gene Therapy INDs (2020)
CQA (Critical Quality Attribute)	A physical, chemical, biological, or microbiological property that must be controlled to ensure product quality.	• Vector genome copies per cell • Identity by flow cytometry • Endotoxin level <5 EU/mL • Residual plasmid DNA levels	ICH Q8(R2): Pharmaceutical Development
CPP (Critical Process Parameter)	A variable in the process that impacts CQAs and must be controlled tightly.	• MOI (Multiplicity of Infection) • Viral incubation time • Culture conditions (O₂, CO₂, agitation) • Washing buffer composition	Bioprocess Intl: Process Validation Strategies for CGT
CMA (Critical Material Attribute)	A raw material property (input) that can affect CQAs and must be monitored or controlled.	• Donor T-cell viability • Vector purity and concentration • Serum or media composition• Bead-to-cell activation ratio	PQRI: How to Identify CQA, CPP, CMA (2015)

This post outlines how QbD applies across four critical stages of process qualification: Engineering Runs, Pilot Runs, Comparability Runs, and Process Performance Qualification (PPQ). Along the way, we’ll explore CQAs, CPPs, CMAs, DoE, and the Control Strategy Document, with examples pulled from real FDA-reviewed BLAs like Kymriah and Zolgensma.

Engineering, Pilot, Comp, and PPQ Runs in CGT Manufacturing

In cell and gene therapy, the transition from development to commercial manufacturing relies on structured process qualification phases. Each phase generates evidence to support a robust, repeatable, and regulatory-compliant process.

Engineering runs are early development-stage batches using non-GMP or development-grade materials. Their purpose is to explore the relationship between Critical Process Parameters (CPPs) and Critical Quality Attributes (CQAs), define initial process ranges, and support assay development. These runs are essential for mapping the process design space and informing risk assessments.

In this phase, you:

• Explore design space through small-scale or non-GMP runs
• Identify relationships between CPPs and CQAs
• Test feasibility of assays, sampling, and tech transfer

Pilot runs simulate full-scale GMP conditions, using qualified materials and final equipment. They verify that the process performs consistently at scale and allow teams to finalize in-process controls, batch record formats, and material handling protocols. QbD Angle: Pilots test the “edges” of your design space — ensuring robustness.

• Use near-GMP conditions and materials
• Confirm SOPs, process timing, equipment readiness
• Finalize critical parameters for full-scale runs

Comparability runs are triggered by process changes (e.g., scale, equipment, material sources). They use side-by-side or matched-run data to confirm that CQAs remain unaffected, as required by FDA comparability guidance. Example: Zolgensma’s BLA included comparability data from multiple manufacturing facilities with matching CQA outcomes and validated assays.

PPQ runs are the final validation stage, often requiring three consecutive full-scale GMP batches. These runs demonstrate that the control strategy maintains product quality across variability in manufacturing.

• Requires ≥3 GMP lots (or justified statistical model)
• Confirms repeatability and control strategy effectiveness
• Includes IPCs, batch records, and assay results

Example: Kymriah’s BLA submitted three GMP PPQ lots that met viability, potency, and purity specs under commercial conditions.

Figure 1 Optimization of the quality by design approach for gene therapy products: A case study for adeno-associated viral vectors https://doi.org/10.1016/j.ejpb.2020.08.002

Design of Experiments (DoE)

DoE is a powerful statistical tool used in the Quality by Design (QbD) framework to systematically investigate how process parameters affect product quality. Rather than changing one variable at a time (OVAT), DoE evaluates multiple variables simultaneously, allowing developers to understand interactions, optimize performance, and build robust manufacturing processes.

In the context of QbD, DoE supports the identification and control of Critical Process Parameters (CPPs) that impact Critical Quality Attributes (CQAs). For example, in cell therapy, DoE might be used to examine how MOI, incubation time, and media composition together influence transduction efficiency and cell viability. Common DoE methods include factorial designs, response surface methodology (RSM), and Plackett-Burman screening designs. These help define the design space—a multidimensional region of operating conditions shown to yield acceptable product quality. DoE not only accelerates development by reducing the number of experiments needed, but it also provides statistically meaningful insights to justify control strategies during regulatory submissions. When implemented early, DoE de-risks scale-up, tech transfer, and PPQ by ensuring that the process is data-driven and predictable, aligning with ICH Q8 and FDA expectations.

The Control Strategy Document

The Control Strategy Document serves as the regulatory and operational backbone in cell and gene therapy manufacturing, aligning your process with quality expectations from agencies like the FDA. Take CAR T as an example, According to the FDA’s “Considerations for the Development of CAR T Cell Products”, sponsors must clearly define Critical Quality Attributes (CQAs) and design Critical Process Parameters (CPPs) to consistently safeguard those CQAs. The document should include:

• In‑process controls (IPCs) and release criteria, such as viability thresholds and transgene expression assays.
• Analytical comparability plans for process or material changes, a key element emphasized in the guidance
• Continuous chain-of-identity and stability assurances, including consistency in leukapheresis procedures, vector titer characterization, and process control strategies

Together, these FDA guidelines inform a Control Strategy Document that integrates CQAs, CPPs, IPCs, assay validation, stability plans, and comparability frameworks—organized to ensure safe, reproducible, and compliant CGT manufacturing.

Figure 2 Control strategy implementation options from doi: 10.1208/s12248-014-9598-3

Final Thoughts

QbD isn’t just regulatory jargon. For cell and gene therapy, it’s your blueprint for making something reproducible from something inherently variable. From first engineering runs to PPQ filing, QbD offers a powerful structure for building—and defending—your process.

If there is a topic that you would like to see here or have a question, please drop us a line at hello@assay.dev

The U.S. Food and Drug Administration (FDA) is a cornerstone of public health, tasked with ensuring that medical treatments are both safe and effective. This vital agency balances its duty to protect consumers with the goal of expediting access to life-saving drugs and therapies. While the FDA is occasionally criticized for its lengthy and costly approval processes, these steps are crucial for maintaining safety and efficacy standards.

In this summary, I’ll briefly walk through the FDA’s drug approval process, the key phases of clinical trials, the types of FDA meetings, and their significance for researchers and sponsors.

What is the FDA Drug Approval Process?

A drug, as defined by the FDA, is any substance intended for diagnosing, treating, curing, mitigating, or preventing disease. It may also alter the structure or function of the body and is often a critical component of medical advancements.

Timeline of Drug Approval

The FDA drug approval process is rigorous, averaging 12 years and over $1 billion from concept to market. It includes:

1. Preclinical Studies: Initial research conducted in labs and animal models.
2. Clinical Trials: A multi-phase process testing the drug in humans.
3. New Drug Application (NDA): The formal request to gain FDA approval for manufacturing and marketing.

Breaking Down Clinical Trial Phases

The clinical trial process is segmented into multiple phases, each addressing specific questions about the drug’s safety and efficacy:

Phase 0: Exploratory Trials

• Purpose: To test the drug in humans for the first time.
• Scale: Very small groups (10–15 participants).
• Key Steps: Use less than 1% of the dose that is expected to have a clinical effect. Requires an exploratory Investigational New Drug (IND) application.

Phase I: Safety Evaluation

• Purpose: Identify a safe dosage range and side effects.
• Scale: 20–80 healthy volunteers.
• Characteristics: Trials often begin with single-dose studies, progressing to multiple doses.

Phase II: Efficacy and Safety

• Purpose: Assess effectiveness while continuing to monitor safety.
• Scale: 100–300 participants with the targeted condition.

Phase III: Large-Scale Studies

• Purpose: Confirm effectiveness, monitor side effects, and compare the drug to existing treatments.
• Scale: 1,000–3,000 patients with the targeted medical condition.

Phase IV: Post-Approval Research

• Purpose: Conduct long-term monitoring after FDA approval to gather additional safety and effectiveness data.

Key Applications in the Approval Process

• Investigational New Drug (IND) Application

Before beginning human trials, a sponsor must file an IND with the FDA. This application provides:

• Drug composition and manufacturing details.
• Preclinical data to establish safety.
• A rationale for testing the drug in humans.

In emergencies, researchers can request an Emergency IND (EIND) to use an investigational drug without delay.

• New Drug Application (NDA) or Biologics License Application (BLA)

Upon completing Phase III trials, a sponsor submits an NDA or BLA to request FDA approval for the drug’s production and sale. The NDA includes:

• Clinical trial data and findings.
• Details about the drug’s manufacturing process and facilities.
• Labeling, risk evaluation, and safety monitoring plans.

The FDA typically reviews NDAs within 10 months. Some drugs may qualify for accelerated review, expediting the timeline to 6 months for therapies addressing unmet medical needs.

Figure 1 Time, Money, and Success: Stages in Drug Development . From here https://doi.org/10.1016/j.jacbts.2016.03.002

Understanding FDA Meetings

Formal FDA meetings play a pivotal role in drug development, offering sponsors guidance and feedback. These meetings are categorized based on purpose and urgency:

Type A Meetings

• Purpose: Resolve issues stalling drug development, such as clinical holds or disputes.

• Timeline: The FDA will respond to a request for a Type A meeting within 14 calendar days, and the meeting will be scheduled within 30 days of the request.

Type B Meetings

• Purpose: Discuss general drug development plans or review data at key milestones, such as end-of-phase reviews.
• Timeline: The FDA will respond to a request for a Type B meeting within 21 calendar days, and the meeting will be scheduled within 60 days of the request.

Type C Meetings

• Purpose: Address any topics outside Type A or B categories, often broader in scope. For example, if the sponsor wants to discuss the development and feasibility of a new biomarker or surrogate endpoint

• Timeline: The FDA will respond to a request for a Type C meeting within 21 calendar days, and the meeting will be scheduled within 75 days of the request.

Type D Meetings

• Purpose: Discuss focused, innovative development questions, typically limited to two key topics.
• Timeline: The FDA will respond to a request for a Type D meeting within 14 calendar days, and the meeting will be scheduled within 50 days of the request.

INTERACT (Initial Targeted Engagement for Regulatory Advice on CBER Products) Meetings

• Purpose: Early-stage discussions for innovative product development. For example, if the sponsor wants to propose innovative manufacturing processes for stem cell therapies or seek feedback on the use of a novel viral vector platform for gene delivery.
• Response Time: FDA The FDA will respond to a request for an INTERACT meeting within 21 days.

Requesting FDA Meetings

Sponsors must submit a written meeting request and package that includes:

• Product details (e.g., name, application number, and proposed indication).
• The purpose of the meeting.
• Proposed questions organized by discipline.
• Background on completed or planned studies.

Meeting packages must be submitted electronically and include a table of contents and additional context for reviewers. If approved, the FDA sends a letter confirming the meeting details, including date, time, and location.

Figure 2. Summary of FDA meeting types. From here https://omarconsultants.com/services/strategic-clinical-design-and-regulatory-meetings/

Meeting Conduct and Documentation

During meetings:

• The FDA assigns a chairperson and reviews the agenda.
• No audio or visual recordings are allowed.
• Sponsors should present clear, focused questions (no more than 10 per meeting).
• Either party summarizes key discussion points and action items.

The FDA’s official meeting minutes serve as the formal record of discussions, agreements, and next steps.

Advisory Committee (AdComm) Meetings

Advisory committees provide expert recommendations on new medical products, focusing on safety, effectiveness, and benefit-risk assessments. These panels, composed of independent experts, offer input that helps the FDA make informed decisions.

Common Delays in FDA Meetings

Delays can occur due to:

• Voluminous or incomplete meeting packages.
• Late submission of additional questions or data.
• Scheduling conflicts with essential attendees.

Sponsors may also choose to cancel meetings if preliminary FDA responses address their concerns adequately.

Accelerating Drug Development

The FDA recognizes the need to expedite drug approvals without compromising safety. Initiatives like accelerated reviews for generic drugs and breakthrough therapies aim to streamline the process, ensuring that patients receive critical treatments faster.

Conclusion

The FDA’s drug approval process and formal meetings ensure a balanced approach to public safety and innovation. While the process can be complex and time-intensive, understanding the structure of clinical trials, key applications like INDs and NDAs, and the different types of meetings can empower sponsors to navigate the system more effectively. With its commitment to efficiency and safety, the FDA continues to be a vital partner in bringing transformative medical treatments to the public.

If there is a topic that you would like to see here or have a question, please drop us a line at hello@assay.dev

Stem cells hold incredible potential in regenerative medicine, offering promise for treating a wide range of diseases. However, turning this potential into reality is a challenging process that involves overcoming numerous technical hurdles, including the need for efficient, scalable, and cost-effective cell production. This is where automation can play a crucial role, enabling researchers to achieve consistent results while scaling up production for industrial and therapeutic applications.

Why Automation in Stem Cell Workflows?

Historically, stem cell research has relied heavily on manual techniques. This requires highly skilled labor and is prone to contamination, variability, and scalability issues. Manual cell culture can be highly time-consuming and labor-intensive, limiting the number of experiments that can be performed and restricting the throughput needed for large-scale research projects or clinical applications. For instance, in the study by Truong et al. (2021), an automated platform (TECAN Fluent) was used to culture and differentiate human induced pluripotent stem cells (hiPSCs) into retinal pigment epithelium (RPE). This automated approach enabled reproducible, and scalable generation of RPE cells, illustrating the potential for automation in personalized drug testing and disease modeling for age-related macular degeneration (AMD) patients. A similar approach can be implemented virtually on any other liquid handling robots such as Hamilton.

The Benefits of Automation in Stem Cell Culture

There are multiple advantages to integrate automation in cell culture workflow which I listed a few key ones with examples from recent publications below:

1. Enhanced Efficiency and Throughput: Automated systems can perform routine tasks such as media changes, cell seeding, and harvesting at a faster pace and with higher precision than manual methods. For example, Bando et al. (2022) introduced a compact, automated culture machine designed for maintaining and differentiating hiPSCs. This system successfully expanded hiPSC cultures under feeder-free conditions and simultaneously differentiated them into cardiomyocytes, hepatocytes, neural progenitors, and keratinocytes. This machine uses a novel x-y-z-axes-rail-system to perform automated cell culture processes, making it easier for research laboratories to adopt iPSC-based workflows without the need for extensive space or specialized training. This level of automation reduces the need for highly trained personnel and supports high-throughput screening in various research fields.
2. Improved Reproducibility and Consistency: One of the main advantages of automation is its ability to reduce variability and human error. This is particularly important in stem cell research, where even small variations in culture conditions can significantly impact cell behavior and differentiation outcomes. Automating these processes ensures that experiments are reproducible and results are consistent across different batches and laboratories.
3. Reduced Contamination and Safety Risks: Automation minimizes human intervention, thereby reducing the risk of contamination and exposure to hazardous materials. This is especially important when working with stem cells for clinical applications, where sterility and safety are paramount.
4. Scalability for Large-Scale Production: Automated systems can easily be scaled up to handle large numbers of cell cultures and experiments simultaneously. This capability is essential for translating stem cell research from the lab to the clinic and supporting commercial-scale production of cell therapies. Elanzew et al. (2020) reported the development of the StemCellFactory, a modular platform that covers the entire hiPSC production workflow, from reprogramming human fibroblasts to expanding hiPSC clones. The platform integrates advanced hardware and software components, enabling parallel processing of multiple hiPSC lines and providing a scalable solution for disease modeling and drug screening.

Figure 1. The StemCellFactory, an automated system for reprogramming and expansion of iPSCs. From DOI: 10.3389/fbioe.2020.00811

Market Trends, Future Directions, and Challenges

As the demand for stem cell therapies and personalized medicine grows, the market for automated cell culture systems is expected to expand rapidly. The global automated cell culture market was valued at USD 18.75 billion in 2023 and is projected to reach USD 47.50 billion by 2032, growing at a compound annual growth rate (CAGR) of 11.28%. This growth is driven by increasing adoption of automated technologies in both research and industrial settings. As automation platforms become more accessible, modular, and user-friendly, they will likely play a critical role in enabling the widespread adoption of stem cell technologies for clinical and commercial applications.

Despite the numerous benefits, there are still challenges to the widespread adoption of automation in stem cell research. Setting up automated systems can be expensive, especially for smaller research laboratories. In addition, operating and maintaining automated systems requires specialized knowledge and skills. Researchers may need additional training to fully utilize the capabilities of these systems. The other challenge is to integrate automation when the lab is already using manual techniques. However, the long-term benefits in terms of reduced labor costs and increased efficiency often justify the investment.

Figure 2. Schematic representation of novel or facilitated applications for hPSCs by automation. From DOI: 10.1177/247263031771222

Conclusion

The automation revolution in stem cell research is transforming the field by enabling high-throughput, reproducible, and scalable workflows. From compact, automated culture machines to large-scale platforms like the BioNex Solution’s Hive and CELLITRO RoboCell, automation is addressing the challenges of manual cell culture and making stem cell research more accessible and effective. As these technologies continue to evolve, they will play an increasingly important role in translating the promise of stem cells into real-world therapies and clinical applications.

If there is a topic that you would like to see here or have a question, please drop us a line at hello@assay.dev

Human embryonic stem cells (hESC) and induced pluripotent stem cells (iPSC) can differentiate into various cell lineages including bone, cartilage, fat, muscle, and neurons. In recent years, there has been growing interest in using hESC and iPSC for developing cell therapies and tissue engineering because of their differentiation capacity. For instance, Fate Therapeutics focuses on cancer and immunological illnesses by developing programmed cellular immunotherapies using iPSC-derived NK and T-cells. Whereas, Sana Biotechnology aims to develop engineered cells as therapeutics using pluripotent stem cells to replace any missing or damaged cells in the body. 

To achieve high reproducibility production, it is critical to monitor and track the migration, proliferation, and differentiation of these cells in vitro (and in vivo). Accordingly, various assays exist to check bio-markers in transcription and translation levels. In this post, we introduce a few most popular methods that are widely used to study genomic expression profiles. These include microarray analysis, RT-qPCR, and RNA sequencing. Each of these techniques offers advantages and has limitations that are briefly summarized in Table 1.

Table 1. Summary of popular techniques that are used to study gene expression profile of iPSC and hESC.

1. Microarray Analysis

Microarray technology, which emerged in the early 1990s, revolutionized genomics by enabling simultaneous analysis of thousands of genes. Microarray was originally developed in two main forms. The cDNA microarrays by Patrick Brown’s team at Stanford and oligonucleotide microarrays by Stephen Fodor at Affymetrix. The latter used synthesized short DNA sequences fixed on a chip, allowing for more precise control over probe placement and a higher degree of miniaturization which made it quickly moved from academic labs to widespread commercialization. Although faced with competition from next-generation sequencing in the 2010s, microarrays have retained a significant role in clinical diagnostics and large-scale studies due to their cost-efficiency and established workflows. Currently, there are several instruments in the market each with unique features and limitations such as Affymetrix, now part of Thermo Fisher Scientific, Nanostring nCounter, and Illumina’s BeadArrays. Regardless of the technology, a microarray workflow typically starts with RNA extraction. Although there are protocols that directly use cell lysis, RNA isolation is advised for better-quality microarray data. Additionally, RNA extraction allows for more robust cDNA synthesis (if needed). Upon isolating RNA (and cDNA synthesis if needed) hybridization is performed where labeled nucleic acid samples (RNA or DNA) are allowed to bind, or hybridize, to complementary DNA probes attached to the microarray chip. There are pre-made panels that contain variant markers relevant to the status of the cell, it is also possible to produce custom panels based on literature. The microarray is then scanned with a laser, which excites the fluorescent dye, allowing the measurement of fluorescence intensity emitted from each spot on the array. The intensity of the fluorescence at each spot on the array is proportional to the amount of target nucleic acid binding to the probe, which reflects the gene expression level of that gene in the sample.

Figure 1. Typical microarray workflow from mRNA to data from here DOI: 10.1128/CMR.00019-09

2. Reverse Transcription Quantitative Polymerase Chain Reaction (RT-qPCR)

The development of RT-qPCR occurred in the 1990s, with the first systems introduced by Applied Biosystems in 1996. This technology integrates fluorescent chemistry to provide quantitative results, enabling the real-time accumulation of amplified DNA during the PCR process. RT-qPCR revolutionized genetic analysis by allowing for precise quantification of nucleic acids, making it invaluable in clinical diagnostics, research, and even forensic applications. Its ability to detect and quantify gene expression and genetic mutations rapidly and sensitively underpins its broad adoption across stem cell research.

RT-qPCR workflow involves (optional) RNA extraction followed by reverse transcribing RNA into complementary DNA (cDNA) and then amplifying specific cDNA sequences using PCR. RT-PqCR is widely used to analyze gene expression in embryonic stem cells and induced pluripotent stem cells. For instance for identification of stem cell markers such as Oct4, Nanog, and Sox2 (pluripotency markers. RT-qPCR is also widely employed to evaluate the purity and quality of stem cell populations by quantifying the expression levels of genes associated with undesired cell types or differentiation stages. For instance, the expression of lineage-specific markers (e.g., hematopoietic markers for mesenchymal stem cells) can be assessed to ensure the absence of contaminating cell populations. The main disadvantage of RT-qPCR is its lower throughput (usually limited to <90 markers) and the fact that it can be affected by inconsistencies in PCR. 

Figure 2. Typical RT-qPCR  from here https://www.americanlaboratory.com/

3. RNA-Sequencing (RNA-Seq)

Initially described in 2008, RNA-seq allows researchers to detect both known and novel transcripts, quantify gene expression levels, and identify isoforms, providing a comprehensive view of the transcriptomic landscape. It involves sequencing cDNA libraries generated from stem cell RNA, allowing for the detection of novel transcripts, alternative splicing events, and gene expression levels. The main advantage of RNA-seq is that can detect both known and novel transcripts, providing a more comprehensive (and individual) view of the stem cell transcriptome compared to microarrays or RT-qPCR. Despite the unparalleled potential, RNA-seq workflow can be lengthy, costly, and it requires complex data analysis pipelines which limit its routine usage. Nonetheless, applications of RNA sequencing and its derivatives such as single-cell RNA-seq is rapidly growing in the field. Refer here for more info doi: 10.1101/gr.223925.117

Figure 3. Typical RNA-seq workflow from from here DOI: 10.1007/s11914-022-00726-x

There are other methods to measure gene expressions such as northern Blotting and Massively Parallel Signature Sequencing (MPSS) that are less common due to their limitations and the existence of better alternatives. In future posts, I will review assays that are available to study hESC and iPSC in the protein level.

If there is a topic that you would like to see here or have a question, please drop us a line at hello@assay.dev

The main event of the Society for Laboratory Automation and Screening (SLAS) is its annual International Conference and Exhibition which rotates between Boston and San Diego. If you are new to SLAS, I highly recommend visiting their expo which you can purchase 1-day or three-day passes for a relatively low price (<300$). As a pro tip, you can most likely get free guess passes from your lab suppliers, especially those related to lab automation or high-throughput instruments. This year (2024) SLAS was February 3-7, 2024 at the Boston Convention and Exhibition Center. https://www.slas.org/events-calendar/slas2024-international-conference-and-exhibition/ I am going to capture a few observations from 1-day my visit here.

Something I noticed in the Expo hall was the growing trend of specialized affordable small liquid-handling robots. These are benchtop robots with less flexibility which are designed to execute a particular task, such as PCR plate setup, and nucleic acid extraction. Macherey-Nagel isoPure mini and MagBinder from Omega-Bio tek are designed for magnetic-bead-based nucleic acid extraction of 1-24 samples. The other system was Myra from BMS which can prep reaction tubes to use with their MIC qPCR. One more flexible system is the Hamilton Microlab Prep. It costs $20-30k and can have up to two independent 8-prob heads. It controls with an integrated tablet and an App-like software. There are also a few devices (heat, cooler, shaker) and accessories (reservoirs, tube, and plate adapters) available that can improve the versatility of the system. It has a limited working deck (9 positions) but I recommend checking it if you are looking to introduce automation to your workflow. It can be a great option for simple tasks such as plate prep, serial dilutions, transfer, and cherry-picking. https://www.hamiltoncompany.com/automated-liquid-handling/platforms/microlab-prep

Figure 1. Example of small footprint robots in SLAS2024 that can simplify low throughput specialized applications.

 

The other interesting innovations were around total lab automation. Biosero Omron is a mobile robot that can move labware between workstations with equipment spread throughout the laboratory, across multiple floors, or even different buildings. Mobile robots can enhance the modality and flexibility of the workflow but would need designated infrastructure and a high level of integration. Nonetheless, it is exciting to see a robot moving a microplate from an incubator to a centrifuge across the room!

There were also a few floating transport systems that combined the advantages of conventional systems and supplemented them with a unique magnetic levitation technology. With their floating 2D transport, they can rotate, tilt, and elevate microplates which opens up a whole host of new possibilities. For instance, Hamilton integrated a Beckhoff XPlanar with a Vantage system (called Vantage Plus) to virtually extend the deck in any direction. Planar Motors was another vendor that offered similar technology.

Figure 2. Two examples of technologies that enable across-lab integration.

 

The other boosts that catch my eyes were CELLTRIO and BioNex which offer highly integrated enclosed robotic systems. These systems are usually cost-prohibitive for most applications. However, they are valuable tools for multi-step applications that are laborious, delicate, and need a sterile environment, such as cell-based processes.  CELLITRO RoboCell Cell Line Automation Platform performs cell maintenance, culture, passage, differentiation, transformation, cell line development, biobanking, assays, imaging, and screening for multiple cell lines such as adherent, suspension, primary, immortalized, iPSC. The platform can handle various flask types, plates, and even serological pipettes. https://celltrio.com/

BioNex Solution’s Hive Automation Platform uses vertical space to facilitate the integration of a large number of instruments and consumable storage in a relatively small footprint.  Vertical design facilitates compact system integration of BioNex instruments with third-party devices without compromising performance. One key technology is their BumbleBee sample handler that provides independent channels and rotating sample translators enabling fast, simultaneous picking from and placing any location on the sample deck.  https://bionexsolutions.com/

Multiple vendors offered OEM services, custom-made instruments/robots, data infrastructure, scheduling software, and more liquid handling system options. One liquid handling system that I saw in several boosts was Lynx from Dynamic Devices. I have not used their system, but seems they are growing in popularity probably due to their Volume Verified Pipetting (VVP) technology. Anyway, it is always enticing to visit SLAS Exhibitions!

If there is a topic that you would like to see here or have a question, please drop us a line at hello@assay.dev

Figure 3. BioNex Solution’s Hive Automation and CELLITRO RoboCell Cell Line Automation platforms offer highly integrative systems for cell and other biological applications.

Okay, you finished your first high throughput screening campaign, and now you need to decide which compounds are standing out for the validation/secondary assays. There are two strategies to select hits. You can either rank the samples based on their effect size in the assay and pick top performers or you can pick samples that meet the pre-set threshold. In either case, the goal is to maximize true-positive rates while minimizing false-negative rates (FNRs) and/or false-positive rates (FPRs). FP means wasting valuable resources in the secondary assay for inactive compounds (false discoveries), and FN means you might miss some valuable candidates in the primary round. Here, I will discuss a few common strategies/methods to identify true hits in primary HTS data while minimizing FNR and FPR.

The first factor that affects the hit selection strategy is whether there are replicates for each compound or not. Because that indicates if we can calculate (and utilize) data variabilities on a sample basis or if we need to assume a sort of distribution. In most primary screening scenarios, and our focus here, there is only one copy for each compound in the screening pool. As a result, in screening without replicates, we rely on the strong assumption of the normal distribution for data variabilities. The second important factor is if we have controls (either one or both negative and positive). The benefit of controls-based methods is that they are straightforward to calculate, and can deal with systematic sources of HTS variability, assuming controls are affected similarly to samples. They can also be used to normalize the data across multiple runs/plates if needed. The following two formulas are some common approaches for hits selection if having controls. The first formula is useful if you want to rank samples and then pick top performers. This is a preferred method for screens with strong control where H and L are the average of High and Low controls. The second is used to test whether a compound has effects strong enough to reach a pre-set level by comparing it to the control value where “std” is the standard deviation of controls and K is an arbitrary multiplier. By changing k, you can make the selection criteria more or less stringer. K =3 is a reasonable choice if the normal distribution is assumed which translates to approximately 95% confidence level.

If there is no control in the screening, we can use the samples, themselves, as controls. “This may seem to be a contradiction, but in reality majority of samples in screening are not active and can serve as vehicle control”1. Formula 3 and 4 are counterparts for Formula 1 and 2 in case there is no control in the test. Substituting the mean with median and Standard Deviation (STD) with the Mean Absolute Deviation (MAD) helps with dealing with outliers and controlling false positive discovery rates. However, MAD gives equal weight to all deviations from the mean, regardless of their magnitude. This means that outliers or extreme values in the dataset have the same impact as any other data point. This can translate to higher FNR. Therefore the choice between the two depends on the distributional properties of the dataset.

These formulas are easy to calculate and interpret. However, they fail to capture data variations and plate-to-plate or other locational biases. As a result, several statistical scoring methods have been developed to address these issues. Probably, the most widely known statistical scoring method in HTS is the Z-score (this is different from Z-factor). It is computed on a plate-by-plate basis, and it is calculated by Formula 5 below. Where μ and σ are the mean and standard deviation of all samples, respectively. You can use the Z-score as a cut-off which in this case ±3 is the common choice if you can assume normal distribution of data across the plate. Alternatively, you can use Z-score values to rank samples, and pick a fixed percentage of the most extreme samples.

Z-score can handle both multiplicative and additive offsets as it has the average on the numerator and standard deviation on the denominator. This is an important feature when processing and analyzing compounds across multiple plates (experimental unit). However, the Z-score still fails to handle positional effects (such as edge, row, and column effects). Its performance is also susceptible to extreme outliers as average and standard deviation are not affected by extreme values proportionally. This can be alleviated by using median and MAD instead. However, they also have their shortcomings. Another strategy is to use B-Score (for “better” score) which might offer better performance than Z-score. The B-score calculation is similar to the Z-score in that they are both the ratio of an adjusted raw value in the numerator to a measure of variability in the denominator. However, both the adjustment and measure of variability are more extensive. These adjustments make the B-score more resistant to positional effects (column and row) and outliers. It also enables combining (normalizing) data over all plates in the screening which might further reduce both false positive and false negative rates. On the other hand, B-Score calculation is more computationally demanding compared to Z-Score which requires specialized statistical software. Check Reference 1 for the full discussion on the B-Score calculations and applications.

A more recent statistical scoring method is the strictly standardized mean difference (SSMD) which was originally introduced for quality control and hit selection in RNAi HTS assays. SSMD can be used to evaluate the differentiation between a positive control and a negative control in HTS assays (QC) or for sample ranking. The SSMD formula typically involves the means, standard deviations, and correlation coefficients between the groups being compared. The standard formula assumes the presence of replicates (Formula 6). In a primary screen without replicates, it can be rewritten as formula 7. Check SSMD Wikipedia for full annotation of each parameter.

As seen in Formula 6, SSMD standardizes the mean difference by dividing it by an estimate of the pooled standard deviation. This standardization process results in a dimensionless quantity that is not influenced by the scale of the original measurements. This makes the SSMD less sensitive to variations in scale and distribution. However, in cases without replicates, SSMD, as well as Z-score and B-Score, relies on the assumption that every compound has the same variability as the reference in that plate. In addition, we may get a large SSMD value when the standard deviation is very small, even if the mean (effect size) is small. Moreover, the SSMD value itself may be less intuitive to interpret compared to other effect size measures. Table 1 serves as a guideline for using the SSMD value for sample classification.

Table 1. SSMD to classify effects are shown based on the population value of SSMD. From Wikipedia

For most HTS cases, the common practice is to combine a statistical scoring method (such as SSMD, B-score, and Z-score) with a controls-based measure (such as average fold change), and then use another scoring method for backup and examination. As such, the dual-flashlight plot can be very useful to visualize and interpret the results. In a dual-flashlight plot, we plot a statistical scoring system (such as SSMD or p-value) versus an effect size measure (such as average log fold-change or average percent inhibition/activation) on the y- and x-axes, respectively, for all compounds investigated in an experiment (figure 1 below). I might have a separate post on the dual-flashlight plot in the future.

In summary, there is a growing list of statistical methods (and hence software) to analyze HTS data and to select hits. Each method has its pros and cons. When selecting a method, it is critical to understand its underlying assumption and the experimental setup such as the presence and quality of controls, replicates, and the nature of screening compounds (small molecules, proteins, RNAi …). A method that works perfectly for small molecule library screening in which strong controls usually exist might fail for screening proteins and RNAi which normally offers a narrower separation window. I highly recommend checking relevant literature to design your HTS data analysis strategy before starting the HTS campaign. It would be squandering to troubleshoot once hits are moved to the secondary assay stage.

Figure 2. Example of a Dual-flashlight plot where strictly standardized mean difference is plotted versus fold change (log 2 scale) in the cytotoxicity assay. The figure is From here DOI:10.26508/lsa.202201605

This post was just an introduction to his selection in HTS. There will be more in-depth HTS data analysis posts here so subscribe to our newsletter now! If there is a topic that you would like to see here or have a question, please drop us a line at hello@assay.dev

References:
https://www.sciencedirect.com/science/article/pii/S2472630322002345
https://www.sciencedirect.com/science/article/pii/S0888754307000079

Imagine you do a high throughput screening campaign, how do you know your assay is reliable enough to generate meaningful hits? Well, one might use signal-to-background (S/B) or signal-to-noise (S/N). However, both parameters fail to fully capture the variability in the sample and the background. Zhang et al. introduced the Z-factor in 1999 to address these sorts of challenges for HT assays https://doi.org/10.1177/108705719900400206

Since the introduction, the Z-factor has become the main statistical measure to assess the quality of an assay. It quantifies the separation or “signal-to-noise” ratio between the positive and negative controls or sample and controls in a screening assay. Z-factor indicates the ability of an assay to discriminate between compounds that show an effect (positive) and those that do not (negative) based on the separation of their distributions.

The Z-factor is calculated as follows:

Where:

∂s,∂c are the standard deviations of the samples and the control, respectively.
μs,μc are the means of the samples and controls, respectively.

∂c and μc are replaced with the mean and standard deviation of positive controls and negative controls for agnostic/activation and antagonist/inhibition assays respectively. A more practical/used/known variation of the Z-factor is the Z’-factor, also known as Z-prime.

While they are practically different, the two terms are used interchangeably often (including in Wikipedia). Z’ is a characteristic parameter for the overall quality of the assay whereas Z is more related to the separation between the signal of the sample and the control. In practice, the assay conditions, such as control selection, reagents, and instruments, are first optimized using Z’. Then, test-compound-related parameters, such as compounds’ concentrations, are tuned by screening a subset of the library to meet Z criteria. The Z-prime (and Z-factor) can take on various ranges and values, each indicating different levels of assay quality:

Z-factor close to 1:

An excellent assay with a well-separated distribution of positive and negative controls. Indicates a robust and reliable assay that can effectively distinguish between compounds that elicit an effect (positive) and those that do not (negative). Typically considered highly suitable for high-throughput screening due to the clear separation between controls.

Z-factor between 0.5 and 1:

Represents a good assay with acceptable separation between positive and negative controls. Indicates that the assay is reliable for high-throughput screening but might have less distinct separation than an excellent assay. Still considered suitable for screening, though with some caution and potential for improvement. This Z range requires strong controls which is usually achievable in small-molecule screening.

Z-factor between 0 and 0.5 (0 < Z < 0.5):

Suggests a less reliable assay with minimal separation between positive and negative controls. Indicates a lower quality assay, less effective in distinguishing between compounds that induce an effect and those that do not. Can be used for screening, but caution is needed, and optimization or modifications may be beneficial.

Z-factor below 0 (Z < 0):

Indicates poor assay performance with overlapping or indistinct positive and negative controls. Suggests that the assay is unsuitable for high-throughput screening due to the lack of separation between controls. Signals the need for significant assay optimization, modification, or potential re-evaluation.

While Z’ is the most widely used QC criterion in HTS, it does have some limitations. The Z-factor assumes that the data distributions for both positive and negative controls (and samples) are approximately normal (Gaussian). If the data deviates significantly from a normal distribution, the Z-factor may not accurately reflect assay performance. This can be especially problematic for small sample sizes (n<30). It is worth noticing that the Z-factor primarily addresses random errors and may not detect systematic errors or biases. In other words, an assay with acceptable Z-factor values may still exhibit systematic errors, impacting the reliability of the results. Moreover, the Z-factor primarily focuses on false positives but provides limited information on false negatives. Some of these can be alleviated by utilizing more robust statistical parameters, such as median instead of average and median absolute deviation (MAD).

Despite these limitations, the Z-factor remains a valuable and widely used metric in high-throughput screening, but it is important to use it judiciously and in conjunction with other quality metrics, such as the dynamic range, assay robustness, and false-positive rates, for a comprehensive assessment of assay performance. It is worth reiterating that the Z-factor may oversimplify the assessment, and a borderline Z-factor may still indicate an assay with useful characteristics especially when screening biologicals. A high Z-factor also does not guarantee the ability to detect biologically relevant compounds (true hits).

This post was just an introduction to the application of Z-factor. There will be more in-depth HTS data analysis posts here so subscribe to our newsletter now! If there is a topic that you would like to see here or have a question, please drop us a line at hello@assay.dev

You can find the Jupyter Notebook that was used to generate the figure in our GitHub repo https://github.com/Assaydev/Z-factor/blob/main/Z-factor.ipynb you can play it with changing average and standard deviations.

Happy Screening!