Are you aware of SAMPL?
Saturday, January 26th, 2008
Greetings,
My name is Aniko Simon, and I am one of the co-founders of SimBioSys. Those of you who know SimBioSys will know that one of our primary areas of focus is virtual screening and we have been developing the eHiTS software for a number of years. Some scientists have referred to us as the “unknown contender” and, unfortunately, it is true to say that relative to some of the other players in the marketplace (for example, CCG, Schrodinger, OpenEye) we are not as well known. We are working on this and this blog is one of our efforts to further our exposure and communication to the market. We hope that the quality and performance of our software speaks for itself however but that needs to be based on a users’ experience and with so many applications to choose from who has the chance to test and compare everything?
We receive a lot of feedback from our users. A fraction of this is made public via publications or talks (http://www.simbiosys.ca/ehits/ehits_enrichment.html and http://www.simbiosys.ca/ehits/ehits_validation.html)
We encourage healthy competition. We have good relationships with the majority of vendors in the marketplace and have much mutual respect for their intellectual prowess and scientific intentions. We enjoy the prospect of comparing the performance of our own solutions with theirs. It is excellent when we have the chance to participate in comparisons. Unfortunately it is all too common to not be contacted by scientists conducting such comparisons and we only get to compare our capabilities with those of other tools after publications are issued.
An example of this situation is described on our website at present (http://www.simbiosys.ca/ehits/ehits_enrichment.html). eHiTS was recently evaluated on 11 targets of pharmaceutical interest in collaboration with Merck Research Laboratories. The original study (without eHiTS) was published last year http://dx.doi.org/10.1021/ci700052x. A later evaluation by Merck of eHiTS on the same set revealed that eHiTS overall performed very well across those 11 families. In particular, it outperformed all the other programs tested on the Carbonic Anhydrase I family. The eHiTS performance average for the entire test set was 18.7, which was the highest average amongst all the tested programs, that included Glide, Fred and Flog.
Just yesterday we discovered another party was going on… ….this one conducted by a vendor. OpenEye are the developers of the excellent Fred docking program (http://www.eyesopen.com/products/applications/fred.html). They are also in the midst of the SAMPL-1 project…the Statistical Assessment of the Modeling of Proteins and Ligands. Details are available online here (http://sampl.eyesopen.com/) but an extract is here:
“SAMPL is a blind test of protein and ligand modeling; an analysis of methods on data not seen by participants. “With SAMPL we intend to avoid ‘who won, who lost’,” explained Dr. Anthony Nicholls, President and CEO of OpenEye Scientific Software. “Rather, we see this as an opportunity for groups to test their methods, learn from the experience and share lessons learned.” In addition, third party testers, particularly professional modelers, are encouraged to participate so as to allow comparisons of single methods when used by several, independent, participants.
SAMPL-1 will consist of two sets of protein-ligand binding data, generously provided by Abbott Labs and Vertex Pharmaceuticals, and sixty-three vacuum-water transfer energies, courtesy of Peter Guthrie at the University of Western Ontario, and Chemical Computing Group. The two sets of protein-ligand data each have between twenty and seventy active compounds, the majority with protein-ligand crystal structures. The assessment will consist of three parts:
1. Virtual Screening.
2. Predicting binding poses.
3. Predicting binding affinities.”
VERY EXCITING!
For those of you interested in this area you can see that such an effort would be of interest to us..and indeed it is! However, we just found out about it yesterday (Jan 25th, 2008) and, based on what we see, everything needs to be submitted by Feb 19th 2008, about 3 work weeks. Quote “The results of SAMPL-1 will be presented at the CUP IX meeting, March 17th-19th in Santa Fe. All participants will be offered a speaking slot at the meeting. The deadline for submission of results is on February 18th, 2008, one month before the CUP IX meeting. For more details, please see http://sampl.eyesopen.com/.”
We’re confused since the announcement went out on Nov 1st to SOME people (we NOW found some blog posts on the web about it). But only became aware of it via these newswire posts yesterday Jan 25th 2008: (http://www.centredaily.com/business/technology/story/355727.html, http://www.newsobserver.com/1566/story/901427.html). Grrrr. Now we have to figure out if and how to participate. This is very short notice for us especially since we have other meetings and tasks lined up already. We’re going to contact OpenEye and ask whether there is a way to get an extension…other informed participants knew the project was coming for 3 months…we just found out and are just not ready. However, we do want to attend the party. It sounds like an excellent opportunity for rigorous examination. We’ll keep you informed about our decision and path forward but either way we encourage you to watch SAMPL-1. The results should be very interesting. Even if we can’t participate we will do a follow up study later when the data are made available.
Watch this space.
