Industry Insights: Examining the Risks, Benefits and Trade-offs of Today’s LIMS
Welcome to the first in a series of Q&A sessions on various aspects of scientific computing. This exchange focuses on laboratory information management systems (LIMS) and the best way to use them. Respondents are key decision makers with many of the commercial LIMS vendors. Among them you will find presidents, directors and product managers (among others). Their responses can be taken as a good index of how their companies view the field.
A few of the responses illustrate paradigm shifts in thinking about the issues involved. I'd suggest you take the diversity of viewpoints as a reminder that the LIMS field is not homogeneous, and that anyone looking to purchase a system needs to perform due diligence and clearly identify their own needs, as well as make sure they understand clearly the context of particular vendors' responses. Putting in this effort will help to ensure a good vendor match and significantly improve your chances for a successful implementation.
Q: In your opinion, what are the biggest benefits to having a LIMS and using it properly?
Huw Loaring, Ph.D.
Howard J Rosenberg
Donna S. Lococo
Randy C. Hice
William S. (Bill) Harten
Huw Loaring: A configurable off-the-shelf (COTS) LIMS can help companies collect more accurate data by removing transcription errors and the need to use multiple spreadsheets, maintain electronic records to good practice (GxP) standards and bring about great time savings by increasing the efficiency of reporting and reducing the amount of quality control (QC) checking required on data.
Tanis MacSween: Recent advances in instrument throughput and data generation rates, core lab consolidation and a focus on efficient lab operations, now make LIMS a requirement for many life sciences labs. The biggest benefits that any life sciences core lab will realize as a result of implementing a LIMS are an improvement in efficiency and traceability in their overall lab operations and data management. If the LIMS is built specifically for genomics, sequencing or proteomics, right out-of-the-box — then the benefits multiply.
LIMS that are designed specifically for life sciences provide added benefits such as: pre-built integrations to many of the most common genomics, sequencing and proteomics platforms and instruments; data management, workflow management and sample traceability to enable efficient lab operations; if the LIMS has a flexible, configurable platform, it also will provide the lab with the ability to adapt to constant changes — technologies, processes, instruments and personnel.
Howard Rosenberg: Benefits from implementing a LIMS and using it properly can be both qualitative and quantitative. Additionally, the benefits are very dependent on the environment within which the LIMS is being implemented. For example, in a pharmaceutical quality assurance laboratory, quantitative benefits can include increased efficiency through the integration of instrument systems, automation of routine reports (COA, weekly, monthly, etc.), automation of calculations and review/approve process by exception only. Qualitative benefits in the same environment would include reduction of transcription error, adherence to regulatory requirements and easy accessibility to data.
Whereas in a research laboratory, quantitative benefits would include efficiency through the integration of instrument systems and adaptable experiment design and workflow. Qualitative benefits would be very much the same as above.
Donna Lococo: It isn’t about data management. It is about managing products, time, resources and risk. So, you need your LIMS to deliver the right information to the right people in a timely fashion. Problems that go undetected can result in expensive product rework or loss. Even the benefits of rapid analysis techniques can be lost if your data delivery mechanisms are not efficient.
Whether a process needs to be adjusted to conform to permit limits or to fine-tune product properties, early detection and resolution of problems directly affects the bottom line. This is where the optimized LIMS provides key benefits, because it doesn’t matter how
- accurate the results are if the sample is not properly identified
- sensitive the instrument is if it isn’t available
- fast the instrument is if you cannot find, use and communicate the results
- good the product is if you didn’t meet the customer’s deadline
Wayne Verost: The benefit of a LIMS depends on the laboratory function:
- Within QC and analytical service labs, the instantaneous evaluation of results versus product specification and statistical quality control (SQC) limits adds efficiency and enhances the quality of the product. Instrument interfaces and automated reporting greatly reduce transcription errors and enhance communication between the lab and its internal and external customers.
- Within research labs, some highly configurable LIMS provide the flexibility to organize work into projects and experiments and include electronic laboratory notebook features such as document (file) attachment of any type. In addition to providing the organization capabilities of a database, the system provides the ability to perform the free-text searching of attachments using a ‘Google-like’ tool that is essential within this environment to greatly reduce duplication of effort.
- For all labs, integrated instrument calibration and maintenance, electronic signatures and audit trails provide automated tools that ensure regulatory compliance.
Trish Meek: One of the biggest benefits to having a LIMS is its ability to integrate with laboratory instrumentation and enterprise systems for better data management. LIMS integrate the various sources of data, including laboratory instrumentation; enterprise systems like ERP, MES and PIMS; and enterprise communication tools, as well as other laboratory tools such as ELNs and document management systems. By doing this, LIMS enable companies to more fully integrate the work of the laboratory into the enterprise.
Tangible and measurable laboratory benefits include data access improvements for external customer queries, simplification and automation of invoicing, productivity improvements, such as using barcoding for sample tracking, the automation of calculations and control of associated results. Additional benefits include more accurate planning and scheduling, eliminating manual transcription and duplication of data, simplifying access to laboratory data and the integration of LIMS with other business systems. All of this translates into time and money saved, efficiencies improved and productivity enhanced.
Bill Harten: A LIMS is like an exoskeleton that protects and harnesses the capabilities of the lab. When properly deployed, a LIMS is a platform for productivity, quality, reproducibility, innovation and control. The LIMS bridges islands of automation with seamless connectivity between key instruments, other software systems, robots. Above all, a LIMS should be the “Commons” for lab professionals to gather and work toward personal, departmental and organizational goals.
A LIMS that is the right fit for the lab will drive the scientific process and will keep track of all of the resources (people, instruments, reagents, etc.) that contribute toward the reliability and quality of the results. A LIMS will keep the laboratory organized and running in an optimal mode for maximum productivity of resources. The LIMS is a comprehensive resource that manages all aspects of the laboratory operation, not singular functions. Biggest benefits: quality, reproducibility, productivity, innovation, profitability and FUN.
John Boother: The benefits will depend on the type of industry and the application. For example, for the contract and veterinary laboratories, the LIMS benefit is that it functions as a business system as well as a system for managing the laboratory. A LIMS should firstly be designed around the reports that are needed from the system to provide the customer with increased information, which can extend to the world outside of the laboratory. Consequently, one of the major benefits is being able to find and report on the data entered quickly and in a multi-dimensional manner. Another major benefit is the traceability and control that a good security system provides within a LIMS. Implementing a LIMS that is designed for change will provide a future-proofed system that will serve a customer for many years, thus avoiding the complications of system replacement.
Randy Hice: In the non-regulated industries, the primary benefits can be traced to productivity increases. These benefits are generally attributed to automated data acquisition, an expedited review and approval process, and a higher confidence in data quality due to fewer transcription errors and automated calculations. In the regulated industries, such as pharmaceutical/biotechnology development and manufacturing, LIMS allows the customer to implement a compliant system, and also to accelerate the submission process through automated clinical functionality.
Q: What constitutes ‘good practices’ with LIMS?
Tanis MacSween: Lab managers today have many choices when selecting a LIMS, from generic LIMS to point solutions, to large-scale enterprise offerings. It is important to be aware of best practices when purchasing a system.
- Know your success criteria and communicate them to the vendors you are considering.
- Assemble a LIMS implementation team at your site and ensure the stakeholders understand the value to them personally and the organization, to ensure a successful implementation.
- Select a LIMS from an organization with domain expertise.
- Select a system with open data access to enable research collaboration and data analysis.
- Select a LIMS with a flexible platform that will adapt to changes in the lab. Labs are constantly evolving — new instruments, new technologies and new protocols are common.
- Instrument integrations are more than file attachments. Find out what vendors mean when they say their LIMS provides instrument integrations!
Huw Loaring: A fully configurable security allowing the development of roles and assigned responsibilities is essential for maintaining good practices within LIMS. A complete audit trail recording the what, when, where and who by of all changes to data, together with an easily searchable report for QA purposes, needs to be available in the software.
Howard Rosenberg: “Good practices” can mean many things in a LIMS environment. It can mean the enforcement of Good Laboratory Practices, Good Manufacturing Practices, as well as the enforcement of FDA and other regulatory bodies’ rules. Most commercial LIMS systems will have the features and capabilities to support these.
“Good Practices” can also mean the proper implementation process and engaging all the stakeholders (management, lab supervisors, researchers, technicians, etc.) to ensure a successful LIMS system. The type of good practice is often the most difficult to attain but is very critical. Implementation processes vary from classic waterfall type processes to agile (prototype) processes and everything in between. Each has its benefits and risks, and which to use is very dependent on the needs of the customer, available resources and culture.
Wayne Verost: A LIMS in and of itself cannot ensure good practices. However, it can provide a set of tools that allow the lab to implement them to the degree necessary to ensure that the data contained with the LIMS is acquired and/or recorded using samples prepared using valid reagents, tested by qualified staff members and analyzed using instruments that are properly maintained and calibrated.
To accommodate this, the LIMS must provide a security system that limits each user’s access to the system pages, functions and data based on their level of training. A version-control system for test methods, product specifications and workflow protocols must be included to ensure that work is properly processed and historically traced. Instrument calibration and maintenance tools ensure that analyses are acquired on validated equipment. Full audit trails and electronic signature capabilities must also be included to accommodate regulatory compliance.
Thomas Kent: Good practices fall into five categories: data management, lab operations, compliance, flexibility and simplicity. A LIMS should
- provide data management that is secure, maintains accuracy, provides real-time access, and allows for searching, reporting and integration with other processes.
- fit with lab operations, including the ability to effectively receive and manage samples information, properly represent locations of samples within containers and transfers between containers, effectively manage storage and retrieval of samples, and allow for handoff of data between processes.
- support various levels of internal and external compliance standards, such as ensuring data consistency, tracking chain-of-custody and enforcing SOP compliance.
- be able to be simply and quickly tailored to lab operations, not force a lab to bend to meet its functionality.
- deliver simplicity, including quick and simple implementation, ease-of-learning for lab professionals, and tailorability that can be performed by either lab professionals or their IT team.
Randy Hice: Good practices are often in the eye of the beholder and a company’s internal policies. From the supplier’s perspective, solid industry-accepted software development lifecycle and strict quality control measures are a basic tenant. Referential integrity in the database protects the customer’s valuable data, and clear upgrade and maintenance policies are also key elements. Last, a progressive, agile approach to product development ensures the product will keep pace with underlying technology such as networks, operating systems, databases and Web services. A key point for suppliers is to generally use development tools that are here to stay, as opposed to one-off developmental tools or programming languages that lack support or an evolutionary pathway.
Bill Harten: “Good practices” is defined by the industry standards to which a lab must adhere (GxP, ISO, CLIA, CAP, AABB and more) in concert with the needs of the lab customers (internal and external). Good practices means tracking all the steps of processes, not only results. Ultimately, the unique requirements of the lab itself must be tracked and controlled by the LIMS. LIMS must support not detract from the organization’s ability to coordinate, communicate, control its lab operation. A LIMS must deliver on each of these core capabilities in order for the evolving dynamics of “good practices” to be met. Absent these key capabilities, focus is directed to figuring out how to make the LIMS work, or manual workarounds to compensate for the shortcomings of the LIMS. In conclusion “good practices” means that the LIMS helps the LAB do its work…better.
John Boother: Most importantly…
- The customer should identify an enthusiastic “LIMS Champion” to lead the project team and be involved with system roll-out.
- Concise user requirements should be defined paying particular attention to work flows and reporting requirements. System users must contribute to this part of the project.
- System design should be user-sensitive e.g. everything on screen should be consistent with the logged users responsibilities and authority — no extras.
- A vendor/customer relationship should be a business partnership.
- The “LIMS Champion” should be allowed to take advantage of user meetings, training and Web-based technical sessions that the vendor offers in order that the implemented system is used to best effect.
- A LIMS needs a powerful security system coupled with audit trails, version controlling of reference data, authentication of key actions (e.g. approval) and individual user auto-log-out times to ensure that all activities are traceable on the system.
Q: What do you see as the biggest benefits and risks of using configurable (change option settings) versus customizable (modify code) systems? How do these compare to custom-built systems?
Tanis MacSween: Some of the risks and benefits that are most important to consider when comparing customizable versus configurable:
-Configurable System Benefits:
- speed of implementation
- adaptable when science/technology changes — your investment is protected
- reduced support costs, because LIMS is scalable across multiple customers and installations
- niche solution very tailored to lab processes
- good if lab has a lot of very unique processes
- takes much longer to implement
- don’t have the ability to adapt to changes in lab
- expensive — TCO is high
Tom Curtis: Configurable systems offer the stability of running on a tested platform. They can be quickly set up to meet most requirements and easily reconfigured to meet changing requirements. All of this reduces the time and cost of implementation and simplifies upgrading. There is a risk that they won’t deliver as much flexibility as the user wants.
Customizable systems also run on a tested code platform. They may deliver more flexibility and more advanced functionality than configurable systems. However, the customization can increase the time and cost of implementation and make it more difficult to upgrade the system.
Custom-built solutions provide the end user with absolute flexibility in functionality. However, they can be expensive and time consuming to implement. End users are dependent on the developers for any upgrades and changes, which may slow response to changes. There is a risk that a custom project may never be successfully completed.
Peter Boogaard: An ideal LIMS is designed with a rich foundation of pertinent functionality, along with easy-to-use configuration capabilities that allow the LIMS to map to unique process requirements, as well as rapidly adapt to changing needs without programming efforts. Due to the static nature of most of the “out-of-the-box” LIMS, users often struggle to enhance their unique process differentiators. On the other hand, LIMS customized to address specific laboratory requirements need significant investment on programming and validation to support evolving changes. Also, customized systems require much longer duration for deployment, leading to higher financial risks.
Keith Wipprecht: There are many benefits associated with configurable systems. In a truly configurable system, all of the functional behavior of the configured software is data-driven and is completely preserved and unaffected by version upgrades. In addition, configured systems are more easily validated and they cost less for the customer to maintain and manage. Most importantly, configurable systems can be adapted and enhanced by the customer, without necessitating the involvement of the software vendor. Considering the maturity and functional completeness of configurable LIMS systems, they present no disadvantage when compared to systems that require customization. With regard to custom-built systems and customizable systems, they are, in fact, one and the same thing. Any software product that is customizable results in an implementation that is custom-built. Such systems are more costly, less compliant and more complex to manage and maintain.
Donna Lococo: Highly configurable systems have a benefit when laboratory operations are dynamic, or the supported business is going through a lot of changes. In these cases, the ability to use option settings to easily accommodate a different workflow, or enable new features, saves time and effort while minimizing the need to restructure existing data in order to support new requirements.
On the other hand, if requirements are essentially fixed and are not subject to much change, there can be a benefit to having a customized module. In an extremely stable environment, a well-designed, coded and tested custom solution can be easier to develop, document and test than one that involves multiple configured options. Many vendors have APIs that minimize the integration risk of custom modules.
A fully custom-built system seems risky unless operations are quite rigid and unique. It goes back to the “good practice” issue — can the system be maintained and supported for its anticipated lifecycle?
Wayne Verost: Customizable off-the-shelf LIMS (COTS) that is fully supported with continuous updates by the vendor is the ideal LIMS. Here, the most detailed functionality is provided to run the business in the most cost and time-efficient manner possible, while taking advantage of core functionality and support by highly experienced professionals. As technology changes, the software will be updated with minimal cost and effort.
Configurable systems are a close second. They are less expensive to implement, but still require a high degree of knowledge to configure and support, and you often must make functional concessions.
Custom-built systems are the worst option. Custom systems don’t include the wide range of ‘core’ functionality provided by OTS /COTS software. Custom systems stagnate soon after becoming operational, are poorly documented and are difficult to enhance once the internal developer moves on to other things (or just gets bored with maintaining the application).
Kevin Cramer: I don’t see much risk to a configurable LIMS. Having consultants come in to program the changes, or changing internal process to adapt to the LIMS is more costly and risky. As far as customizability, it really depends on the details. An easy-to-use API with plug-ins that extend functionality may not alter the core code at all and, therefore, presents no risk. If core code or the DB schema is being altered, then the risks are greater.
The most important point on this can not be overstated — no matter how you are configuring\customizing the LIMS, your changes should be automatically carried forward when the vendor ships you a new release. Many times, these customizations lead to a “one-off” version, which means that, when the vendor ships a new release, your changes no longer work. This can literally require months, if not years, of consulting work to upgrade your changes to work with the new release.
Thomas Kent: When a LIMS has the ability to be tailored (configured), as compared to requiring customization (code modifications), lab professionals can directly adjust the LIMS to meet the needs of their operations and processes without requiring expensive and time-consuming technical resources. SciLIMS, as an example, can be implemented in a lab within the same day of getting started and allow the lab to get meaningful results the same day (as compared to requiring months of elapsed time for implementations needing customization). This dramatically reduces implementation costs, as well as total cost of ownership. Custom-built systems provide the most flexible approach for delivering LIMS features, but this approach is even more time-consuming and expensive than customizable LIMS; additionally, custom-built LIMS projects traditionally exhibit a high rate of failing to be delivered within scope, on time, and within budget.
Randy Hice: Very few top-tier LIMS allow code modification for a number of reasons. This is a great risk to the ability to upgrade systems and to maintain a supportable environment. Most mainstream LIMS suppliers allow the system to be tailored to a customer’s environment through configuration. Custom-built systems are never a good idea, as the customers or small consulting shops producing them design them to address a specific need at a specific time. However, as the business changes, these systems are rarely easy to upgrade and support.
Colin Thurston: Customization is ANY manually written code that modifies system behavior. Whether the LIMS embeds a scripting language or requires custom functionality in an external tool or environment, any written instructions to create functionality represent customization — at additional cost in time, money and resources to the customer. This would include manually creating HTML for Web-based user interfaces, or writing stored procedures to automate workflow processes. Configuration, on the other hand, offers control over the software without requiring any additional code. The special challenges facing different industries can be ideally addressed with solutions that deliver as much application-specific functionality as possible out-of-the-box to meet the particular needs of each laboratory. When functionality is built into the base system as standard, the laboratory has benefited by not having to develop lab-specific customization during implementation, which can significantly reduce implementation and validation time, installation costs, and the pain of future upgrades and requisite validation.
Q: What are the various options for data interchange with other systems? Please discuss the relative trade-offs of each. Which ones do you prefer to use/recommend?
Huw Loaring: Data can often be imported using simple file exports in csv or txt format from legacy instrumentation. However, the preference is to develop direct database links between instrumentation and LIMS or to allow data to be collected directly using simple RS232 output from instrumentation, such as balances.
Tom Curtis: One option for data interchange is with instruments and instrument data systems. The key is to select a solution that will provide a high level of automation and security and the flexibility to easily adjust to changing instrumentation and analytical requirements. A configurable solution that can interact directly with both instruments and LIMS and provide additional calculation and data management capabilities will deliver the most value to the lab.
Another opportunity for data interchange is between LIMS and SDMS. Ideally, someone looking at sample results in LIMS should be able to know exactly where to find the raw data in SDMS that supports those results, and someone looking at raw data in SDMS should have the information they need to find the reported sample results in LIMS. This type of connection between LIMS and SDMS can be created as part of an automated data collection and reporting interface.
Keith Wipprecht: The most powerful of the numerous options for data interchange is Web services. Another approach is record importing and exporting, based on established industry standards, such as XML, HL7 and ASCII text file formats. Other options include: database to database interaction via SQL statements; direct integration with Microsoft applications via OLE automation; and the use of specialized software capabilities for instrument integration. Each option has its appropriate application, and may be the one considered most suitable depending on the circumstances. Nevertheless, the power of Web services cannot be understated, nor can the simplicity of standards-based data exchange.
Howard Rosenberg: Today, there are a myriad of ways to exchange data or integrate to other systems. The preferred methodology entails the utilization of application programming interfaces (APIs). Of course, in order to do this, both systems should have supported APIs. Utilizing this methodology will generally ensure that not only will interchange of data be enabled, but the security systems in each system will be engaged so data access rights and functionality restrictions are enforced. Another option is to directly integrate through the databases of each system. Systems often will provide an ODBC or JDBC driver to allow this to occur. While this methodology gives great data access, it can allow access to data without proper security checks, depending on where the system’s security and rules reside. A further option is to use referencing of tables and fields across SQL compliant databases. Again, one must be cognizant of possible security issues in this methodology.
Donna Lococo: I’d rather see true dialog than data interchange, because operating in real time is such an important factor for decision support. But there are a lot of challenges involved. The most robust interchange formats are usually the most specific — very much built for purpose. The problem being that the mechanism you use for your ERP isn’t going to work for peer-to-peer exchanges, or handle instrument data, or automate regulatory reporting/notification. One size does not fit all!
This leaves me thinking that, as vendors, our focus isn’t really the data interchange standards themselves. We don’t get to control that — we just have to be ready to respond to whatever combinations exist for our clients. Our efforts need to revolve around creating translation or mapping layers within our applications, and utilities that help us automate detection and reaction to trigger activities. This approach will allow us to use the best standard for the task at hand.
Kevin Cramer: This can be a real problem. Certain sectors are trying to establish standards for such interchange, like the clinical data interchange which is XML based but, for the most part, there is no real set of standards for cross-application communication. That being said, if a product has a well-conceived and defined API, then it should be easy to integrate with that application. Unfortunately, most software vendors relegate the API to the bottom of the list when it comes to designing their application, so it is surprisingly rare to find a good API with which to work. This means that we often are using flat file importing from other apps, or we go directly against the other applications DB to get what we need.
Thomas Kent: There are three fundamental methods for interchanging data between LIMS and other systems:
- manual data interchange
- import/export using standard tools
- service APIs
For labs with minor data interchange needs, a manual data interchange process may be sufficient, whereby lab operators manually enter data across systems or merely copy data files. The most common form of data interchange is to utilize import and export features of systems based upon standard tools, such as Microsoft Excel, or standard metadata formats, such as XML. The data exchange approach that supports the highest level of automation is the use of service APIs (application programming interfaces), whereby systems expose programmatic access to read and write data through the use of programming tools. The most appropriate form of data interchange for a given lab depends on the lab’s data load and willingness to automate their data interchange processes.