Home > Informatics > Selecting the Right ELN in 5 Steps

Selecting the Right ELN 
Proven, 5-step screening reduces search time to pinpoint the most suitable solution

Michael Rubacha and Anil K. Rattan, Ph.D.

View PDF version   

click to enlarge  Figure 1

As electronic laboratory notebooks (ELNs) become increasingly popular, the number of vendors steadily increases and, at the same time, finding the right ELN for your company becomes more difficult. This article introduces a proven, five-step screening process capable of reducing search time and enhancing your ability to pinpoint and implement the most suitable ELN to meet your needs. The process outlines a method for quickly accumulating potential vendors; sifting through company startups, shutdowns and mergers; and effectively eliminating vendors based on business needs.

During an ELN case study, performed at Bristol-Myers Squibb (BMS) employing this process, 42 potential vendors were identified, 29 percent were sifted as startups, shutdowns and mergers; and 40 percent were eliminated based on business needs. The remaining 31 percent of vendors were scrutinized against the company’s ELN requirements, resulting in the additional removal of all but three ELNs from consideration. As seen from the case study, this screening process drastically reduces the lead-time associated with selecting the right ELN for implementation.

Pharmaceutical companies are constantly trying to implement new workflows, technologies and paradigms to work and think outside the box. Most recently, a paradigm shift has occurred in the arena of laboratory notebooks. ELNs have begun to replace paperbound notebooks as the normative source of information recordkeeping. This shift has so inundated the pharma industry that, to date, ELN is no longer considered exceptional but rather mainstream, as each of the top five big pharma companies, according to 2008 standings, are now making use of these technologies.1

According to Michael Elliott, CEO of Atrium Research, “ELN technology, used only by early adopter and technology-enthusiastic end users as little as five years ago, has risen to become a mainstream laboratory informatics platform.”2 The recognition of ELN technology as a viable alternative to paper has resulted in a market boom of potential solutions. According to Scientific-Computing’s Laboratory Informatics Guide 2009 section entitled “ELN suppliers,” 34 solutions are currently available for consideration.3 This can be a daunting number for the potential customer. So, how does a pharma company sift through a pool of vendors to choose the ELN that best suits their needs? By employing an effective selection screening process.

Effective screening eliminates the need for in-depth review of every available vendor. Instead, vendors are sifted through a process designed to remove unsuitable choices, outlined as follows:
1. Accumulate an initial vendor pool.
2. Determine a definition for ELN that meets your company’s needs.
3. Compare vendors against the definition.
4. Distribute a vendor questionnaire.
5. Rank the returned questionnaires, and select the top vendors.
These five steps for screening streamline notebook selection searches and should be considered for use during any ELN selection process. 

Methods
Step 1: Vendor Pool
The five-step screening process begins with the accumulation of potential vendors for review. In an age of information-at-your-fingertips, the internet has proven to be a powerful research tool for ELNs. Google searching,4 online informatics sites and vendor news releases all serve to familiarize a customer with the available pool. For example, a Google search on “electronic laboratory notebook” is a good place to start for the uninitiated ELN customer, as it lists a handful of ELN reference information sites, as well as various vendor Web sites.
After persuing Google results, the Atrium Research Web site should be considered for expanding your initial Google pool.5 Atrium Research lists classic ELN vendors, as well as additional ELN-like vendors and intellectual property protection products, each with their respective Web site Continuing the accumulation of your vendor pool, a review of the Laboratory Informatics Guide 2009 section entitled “ELN suppliers” contributes additional vendor solutions not found at the Atrium Research Web site. Lastly, probing through vendor news releases will reveal the current industry usage of certain vendor ELNs.

Step 2: ELN Definition
Step two of the screening process is no small task. Determining an ELN definition is not as easy as it sounds. An adequate definition is paramount for the removal of unsuitable solutions, while retaining those vendors that will ultimately fulfill your company’s requirements. A good approach to take when developing a definition is to leverage the terminology currently being used in the field.
Some suggested sources to help formulate generic and specific ELN definitions are: “Are ELNs Really Notebooks,”6 “Roadmap to a Clear Definition of ELN,”7 and the international Collabora-tive Electronic Notebook Systems Association (CENSA) definition.8

Step 3: Solutions vs. Definition
Once a definition has been formulated, the initial removal of ELN solutions can begin. At this point, any solution that loosely fits your definition should remain in the pool, while those that grossly miss the mark should be excluded. The reason for this is simple: The customer shouldn’t be the one doing all the leg work. Let the vendors give their best pitches as step four of the screening process commences.

Step 4: Vendor Questionnaire
The vendors that at least loosely fit your ELN definition should now be contacted for the first time. Contact can be as simple as an e-mail noting interest in the vendor solution, or more involved with a phone call to each vendor’s sales department. Please note, at this point there are still a large number of solutions left in the pool, and e-mail is a more expedient way of contacting each vendor rather than by telephone.
After extending an invitation to communicate with each vendor, a reasonable internal deadline should be set for their replies. For those vendors that respond to the initial contact, a questionnaire should be sent for completion in order to evaluate each vendor’s fit against your company requirements. This questionnaire should be thorough enough to eliminate any vendors that remain in the “loosely fitting” category of the pool.
Another reasonable deadline should be set for the questionnaire and should be clearly communicated to the vendors as the questionnaire is sent out. The vendor’s ability to meet deadlines is a good indication of their ability to consistently meet your needs.

Step 5: Ranking and Selection
Review of returned questionnaires should be the first time your company performs an intensive look at vendor solutions. Until this point, the screening is meant to expedite the excessive time required to perform an exhaustive review of available notebook solutions.

The returned questionnaires should now be scrutinized for content and completeness. Once the questionnaires are deemed acceptable or complete, the vendors must be ranked. An efficient way of ranking these vendors is through weighting. Each question within the questionnaire is given a weight determined by its individual importance to the project. Each vendor answer is also given a weight determined by the ability of the solution to meet your company’s requirements.

A simple weighting scale ranks questions from 1-3:
1: nice-to-have
2: recommended
3: core requirement

Answers are weighted 0-3:
0: unacceptable
1: requires third-party product
2: requires customization
3: acceptable

Each weighted question is multiplied by each weighted response in order to give the individual question score. The summation of individual question scores yields the total vendor score for a given solution. The highest vendor scores correspond to the top-ranked vendors. These top-ranked vendors should be considered as the best equipped to meet your company’s requirements for ELN implementation.

click to enlarge  Figure 2

Results
A BMS case study performed September to December 2008 made use of this screening process to evaluate its effectiveness.

Step 1: Vendor Pool
A Google search for “electronic laboratory notebook” resulted in the selection of 10 vendors, listed in Table 1 under the “Google” resource category.
The Google search also included a link to the Atrium Research Web site. This resource provided a table of 28 classic ELN vendors, as well as tables for 14 ELN-like vendors and 12 intellectual property protection products. Only the 28 classic ELN vendors were considered from this resource. Of these 28 vendors, 19 were not part of the initial Google pool and were subsequently added. These additional vendors are listed in Table 1 under the “Atrium Research” resource category.
At the time of this case study, Scientific-Computing’s Laboratory Informatics Guide was in its 2008 edition.9 This edition’s section “ELN suppliers” generated 36 ELNs resulting in an additional 13 vendors to the Google and Atrium Research pool. These additional vendors are listed in Table 1 under the “Laboratory Informatics Guide 2008” resource category.
This brought the total pool volume up to 42 vendors. For the remainder of this article, ELNs will not be listed by name in order to ensure that each vendor be given fair consideration during future screening processes by potential clients. With the initial pool accumulated, an adequate definition for ELN was then formulated.

Step 2: ELN Definition
The BMS case study chose to leverage the CENSA definition for ELN, due to CENSA’s expertise as “an international industry association and consulting firm, focused on driving the state of the art for electronic records systems and collaborative informatics technologies in software end user and developer organizations”8 and to BMS’ end user membership in the organization. The definition states, “An electronic notebook is a system to create, store, retrieve and share fully electronic records in ways that meet all legal, regulatory, technical and scientific requirements”.8 The requirements mentioned in the definition were subsequently defined by BMS as the screening process entered step three.

Step 3: Solutions vs. Definition
As vendor solutions were reviewed against BMS definition requirements, several ELNs were found to be discontinued, merged with other ELNs, or inaccessible via internet searches. These ELNs constituted 12 percent (five ELNs) of the initial pool. Also from the initial pool of vendors, 31 percent (13 ELNs) were found to be lacking BMS definition requirements. This is not to say that these vendor ELNs are unsuitable for use, but rather that they did not meet the requirements of this case study.
The remaining 57 percent of vendors were then contacted via e-mail.

Step 4: Vendor Questionnaire
A generic e-mail message was sent indicating BMS’ interest in each vendor’s respective ELN. An internal deadline of two weeks was set by BMS for vendors to respond. This time period was established based on BMS’ project deadlines.
Seventeen percent of the initial pool (seven ELNs) failed to respond to the sent e-mail. During the two-week response period, a 12-page questionnaire was generated detailing BMS’ requirements for a prospective ELN. An outline of the major sections of this questionnaire is shown in Figure 1. The prepared questionnaire was sent via e-mail, as an attachment, to those vendors that began a dialogue with BMS. Each vendor was notified in the e-mail of a six-week deadline for completion and return of questionnaires.
Of the initial 42 vendors, nine percent (four ELNs) either chose not to respond or missed the six-week deadline. Those choosing not to respond cited the content of the questionnaire as the reason for not further pursuing BMS’ business, i.e., they couldn’t meet the business needs outlined in the questionnaire. This left only 31 percent of the initial 42 vendors for step five. An overview of the screening process results is shown in Figure 2.

Step 5: Ranking and Selection
Returned questionnaires were perused to ensure completeness. It was assumed that any uncertainty about the meaning of a question was the sole responsibility of the vendor to rectify. This assumption was indicated in the e-mail sent along with the questionnaire.
A team of BMS subject matter experts was assembled to weight each question and answer in order to accurately rank each vendor ELN. Although time-consuming, this step cannot be rushed and should not be overlooked. The team’s meticulous weighting of questions ultimately highlighted BMS’ core requirements while minimizing the “nice-to-haves.”
The subject matter experts identified 24 percent of the initial pool (10 ELNs) as low-scoring. The remaining seven percent of vendors (three top-ranked ELNs) were then notified of BMS’ interest in viewing product demos and moving forward with negotiations.

Conclusion
Sixty-nine percent of the initial pool (29 ELNs) was eliminated with minimal BMS effort by employing this screening process. Of the remaining 13 ELNs, all but three were removed from consideration after the scoring procedure was applied.

Of the five steps listed, it is recommended that the majority of client effort be applied to steps 4 and 5 while steps 1 through 3 need only require a few days to complete. This will ensure that the most adequate ELN to suit your company’s needs will be selected.

This screening process drastically reduces the need to perform time-intensive searches when attempting to select the right ELN for implementation. The process is effective because, invariably, some companies will fail to respond to given deadlines or will opt out after reviewing client requirements. Adherence to this screening process was proven effective for BMS during its case study.

Post-screening recommendations: After screening has been completed, the top-ranked vendors should receive a request for proposal (RFP) which includes a user requirement specifications section. Concurrent with the RFP should be a multi-vendor pilot process. During this process, your company should be allowed to product test each of the selected ELNs. Product testing should involve subject matter experts, as well as potential end users. Upon completion of the RFP, a refined weighting process should be applied and the final vendor chosen based on overall RFP score and product testing results.

The authors acknowledge Thomas Franceschini and Harbhajan Dhillon for their contributions in editing this article. Special thanks to the vendor representatives who assisted in providing clear and concise information about their respective ELNs.

References (accessed July 2009)
1. SAFE-BioPharma Association. www.safe-biopharma.org/Files/CollateralBrochureFinal.pdf
2. Elliott, M.H. "Electronic Laboratory Notebooks Enter Mainstream Informatics." www.scientificcomputing.com/Electronic-Laboratory-Notebooks-Enter-Mainstream-Informatics.aspx
3. Laboratory Informatics Guide 2009. content.yudu.com/A107dt/LIG2009/resources/index.htm?referrerUrl
4. Google Search Engine. www.google.com
5. Atrium Research and Consulting. www.atriumresearch.com/html/eln.htm 
6. Elliot, M.H. Are ELNs Really Notebooks? Scientific Computing & Instrumentation. July 2004. www.atriumresearch.com/library/July%202004.pdf 
7. Hice, R.C. Roadmap to a Clear Definition of ELN. www.scientificcomputing.com/article-in-Roadmap-to-a-Clear-Definition-of-ELN-051509.aspx 
8. www.censa.org/censaorgfiles/framedef/f_public_resources.html
9. www.scientific-computing.com/lig2008/eln-companies.htm
10. www.amphora-research.com/products/openeln.html
11. www.contur.com/products/products.asp
12. www.idbs.com/ELN
13. www.ipadeln.com
14. www.labtrack.com
15. www.notebookmaker.com
16. www.rescentris.com
17. sourceforge.net/projects/eln
18. www.symyx.com
19. www.velquest.com
20. www.agilent.com/chem/ELN
21. www.artuslabs.com
22. Axiope.com, www.axiope.com/pages/index.php
23. www.cambridgesoft.com/solutions/KnowledgeManagement
24. www.cheminnovation.com/products/cbis/cbis_modules.asp
25. www.deltasoftinc.com
26. www.edge-ka.com
27. www.enso-software.comWebSite2005/Default.aspx
28. www.ezquant.com/en
29. iwww.iadvantagesoftware.com/wpesoverview.asp
30. www.invent-software.com
31. www.kinematik.com/pages/solutions/eln.html
32. www.labnotes.com
33. www.paperlesslab.com/default.asp
34. www.quattro-research.de/cms/Products/ElectronicLabNotebookquattroLJ/tabid/56/language/en-US/Default.aspx
35. rwww.redesignlive.com
36. www.plm.automation.siemens.com/en_us/industries
37. www.tripos.com/index.php?family=modules,SimplePage,,,&page=lab_informatics
38. www.waters.com/sdms
39. arraygenetics.com/home.php?contentBlkName=Products
40. www.ekmco.com
41. www.elevate.se
42. www.mdl.com
43. www.e-nnovate.com)
44. www.evernote.com
45. Iwww.infotrieve.com
46. www.researchnotebook.com
47. www.macinchem.fsnet.co.uk
48. us.mt.com
49. www.studylog.com
50. www.textco.com/products/GI16/gene_inspector.html
51. www.ugs.com

Michael Rubacha is an associate scientist and Anil Rattan is a scientific information manager at Bristol-Myers Squibb. They may be contacted at
editor@ScientificComputing.com.