Scalability, performance and change control... Part 5 of Questions to Ask Your eClinical SaaS Vendor.

In this fifth and final installment of the series on “Key Questions to Ask an eClinical SaaS Vendor” we will consider three topics identified as important to the short and long term effectiveness of Software-as-a-Service (SaaS) based systems: scalability, performance and change control.

Thanks to all who responded to our survey about the eClinical provider topics that are most pressing to you.  We found significant interest in the three topics covered within this post.  We also found significant interest in the cost of eClinical systems (see our “True Cost of CTMS” report) and eClinical integration (which we will cover in the near future).

Scalability

If your organization is planning for growth among your study management team(s) and/or study portfolio, it is critical to consider how the eClinical provider will “scale up” to grow with you and continue to provide the services essential for your success.  Scalability is also a key factor for organizations that prefer to adopt a system gradually over time (thus scaling as they add users and studies).  This type of scalability includes the system’s ability to handle significant increases in user activity and volume, as well as non-technical aspects like cost.

Performance

Web applications have the convenience of being accessible from any computer (or device) with an internet connection and browser.  However, many of us have experienced slow websites, and it can be difficult to determine whether the problem lies with your computer, the server, or something in between.  It can be a challenge, but a capable provider should employ a number of techniques and tools to identify and minimize performance issues.

Change Control

Since SaaS-based applications are hosted and maintained directly by the provider, changes to the application or database (e.g. enhancements, customizations, patches, fixes, etc.) are also controlled by the provider.  Such control has pros and cons for the customer.  The SaaS model can allow an innovative provider to deliver enhancements and improvements quickly and seamlessly.  However, it can also be frustrating when your commonly used features are moved or changed without your team’s prior knowledge.  A basic understanding of the provider’s change management process will help you determine if the provider’s objectives match your own.  Changes to the system may also have validation implications.  Validation is a complex topic that goes beyond the scope of this post, so we encourage you to discuss validation and compliance directly with the vendor.

Questions To Ask (Scalability & Performance)

Following are some questions you can ask of an eClinical SaaS provider to learn more about their processes, priorities and controls.  Let’s start with scalability and performance:

  • Will your system scale to meet our growth?
    How will the provider ensure that the system is expandable, or is currently capable, of handling your future growth?  A provider that plans for scalability should have a number of options to ensure your future needs will be met.  Adding more servers (or larger servers) is one common option, but there are many ways to address scalability.
  • Does the cost of the system scale up and down based on our needs?
    What if you want to start with a small team, and ramp-up your adoption of the software over time? Are there additional costs if you add studies, sponsors or outsourcing teams?  What if you want to or need to scale back your use of the software?  Will the cost scale down as well? 
  • How do you monitor the system to detect performance problems?
    With all the benefits of web-based applications, there are also many points of failure that can materialize as a performance problem. So what types of proactive monitoring does the provider employ to detect performance problems?  A capable provider should be monitoring more than just the servers. 
  • Is performance and scalability part of your software quality assurance plan?
    Although software QA may often focus on functional testing (e.g. testing features), testing scalability and performance is critical as well. Scalability testing often involves testing with large data sets (e.g. large number of sites, subjects, etc.).  Performance testing often involves simulated users and benchmarks for response time.  Many best practices and tools exist to aid in these types of testing. 
  • How do you handle performance for a global team?
    If your user base is geographically dispersed (e.g. multiple countries and/or continents), it is common to be concerned about differences in performance from different locations.  Ask whether your provider has a strategy for optimizing the performance for your user base.  This may be as simple as having flexibility to choose a data center location (e.g. US vs. Europe vs. Asia) that will provide the fastest access to the bulk of your users.  Providers can also employ monitoring tools to watch access times to the application from many locations throughout the globe.

    Questions To Ask (Change Control)

    • How frequently do you update the product?
      One benefit of the SaaS-model is that it allows the provider to innovate and improve the system with frequent updates and minimal impact to the user base.  However, consider whether the providers release cycle is a good fit for your organization’s objectives.  You may appreciate a system with a frequent improvement cycle, or require the stability of a more consistent system.
    • Do you have an emergency fix process?
      If a problem is found that is blocking your ability to use the software, how quickly is the provider likely to respond?  An agile yet disciplined approach often provides a good balance of quick response with improved stability.
    • How do you notify users that a change has occurred?
      When a significant software update has been released, it is nice to know if changes have been made (for better or worse) to the features most important to you.  How will you know that a change has been made that is relevant to your organization’s use of the software?
    • Do you seek user input for enhancement/release planning?
      A collaborative engagement between provider and users (perhaps the key stakeholders for an organization) can be extremely beneficial to ensure an effective application that meets your current and future needs.  Consider whether the provider has a process for including user-based feedback in the planning cycle, and whether the provider previews upcoming features to relevant clients.
    • How do you handle client-specific changes?
      Due to the wide spectrum of life science organizations with eClinical needs, many providers offer client-specific enhancements or customizations (e.g. reports, trackers, etc.).  Does the provider support customizations when needed?  Does the vendor have a strategy to keep client-specific functionality isolated?  You may not want other organizations to see your customizations, nor would you want to see theirs.

    Conclusion

    There is no set of “right” answers for these questions that are suitable for every customer.  However, the questions are intended to help you profile, compare, and determine if a product and provider are a good fit for your objectives. 

    At Trial By Fire Solutions, we are happy to further discuss our strategies for scalability, performance and change-control for SimpleCTMS: our flexible, secure, SaaS-based Clinical Trial Management System (CTMS).

    For a product demo, or to find out more, please contact us.  As always, we welcome your feedback.   Stay tuned for the next post in this series, where we will discuss eClinical change control procedures.

    Simple TrialsClinical Trial Management Systems, CTMS, Clinical Trial Management, eClinical, SimpleCTMS