Patrick Oot, director of electronic discovery and senior litigation counsel at Verizon, recently posted an article, “Keeping EDD In-House Could Contain Costs,” which is insightful on the advantages of moving E-Discovery in-house, highlighting both cost savings and responsiveness. At a time when more vendors are offering SaaS and cloud-based computing, there are still compelling reasons to consider an in-house solution. Verizon’s current in-house project is forecast to save $4 million in one year and provide a 350% ROI over three years. Near the end of the article, Patrick makes a point which I thought bears repeating, namely the challenge of dealing with enterprise software vendors.

‘If you do decide to use a vendor, keep in mind that the promises from many software providers echo the puffery and baiting of snake oil salesman Tobias in “Sweeney Todd.” I wasted six months vetting a Socha-Gelbmann top five “pants-on-fire” software provider that flat-out misrepresented their capabilities.’

The Socha-Gelbmann Survey website now has a disclaimer warning users not to rely too heavily on their survey results.

The reality is that Patrick’s experience is common in enterprise software; however, he is lucky in that he is deploying another solution that will be ready in 2009. With high margins and lack of information, customers are often required to make purchase decisions without full information. The high ticket value of these software sales means that many organizations simply cannot afford to buy another solution if the one they chose does not work for them. It may be years, before a replacement can be considered.

Customers can find themselves in this situation due to any number of reasons, but a primary one, is the lack of information. Often times, customers are at a major disadvantage when making a purchasing decision. One issue is that there is not enough discussion on how products actually perform in production environments. A reason for this may be that it is difficult to change vendors and customers do not want to upset the one they are already in bed with, especially if they cannot move any time soon. Enterprise software isn’t like automobiles where people can be very vocal about their opinions and experiences.

It is challenging enough to choose the right solution given that it is often not possible to evaluate solutions in a production or near-production setting, but as Patrick mentions, some vendors are outright dishonest. This can occur either verbally or technically by using slight-of-hand product demonstrations.

When it is difficult to do an evaluation and also difficult to get straight responses from vendors, it can be useful to talk experts. In the technology industry, there are analysts. However, one must consider how much actual experience analysts have with products (“achmed” posted an observation in a blog comment a while back in July). Given that many analysts do not use the multi-million dollar products they recommend, the reliance customers place on them shows just how great the need for reliable information is.

At the end of Patrick’s article, he provides good guidelines for choosing vendors. The situation requires the customer take responsibility to perform the proper due-diligence, especially given the costs of choosing the wrong solution. Some of his recommendations include:

  • Enlist an infrastructure expert who can work with sales engineers to spec a system adequate for your needs. My team relies on our Verizon Security team for its infrastructure expertise and software evaluation ability to build the system.
  • Ask for specific proofs-of-concept with your own data set. Do not accept demonstrations on Enron data. Or even better, ask for a pilot installation.
  • Anyone can dream up vaporware. Assure that you are not paying a vendor’s development costs as guinea pig number one.
  • Ask for as many references as you can. The PowerPoint slide that lists a provider’s clients doesn’t necessarily mean those clients are using the tool you are considering. Ensure the customer base is actually using the software your team is looking at.
  • Don’t accept “proprietary” as an answer. You should understand the software as if you were to answer in-depth questions about the process during an adversarial deposition.

Regarding references, as important as it is to hear from pre-selected happy customers, one can learn a lot by talking to customers that have not been provided as references. Depending on the industry you are in, you should be able to find peers that have experiences running the products you are considering.

Choosing and purchasing enterprise software is expensive, time consuming, and challenging. By doing the proper research and understanding the quality of data you are receiving, you can maximize your chances for success.