E-mail
Attending: Joel Farrell, Chair; Dan Rehak, Valerie Smothers
Guest speaker: Doug Davis
Dan - a number of organizations are building web services. Both IMS and MedBiquitous have profiles of best practices. In a meeting in Oct, some folks suggested harmonizing those best practices. Chuck Allen recommended contacting Doug Davis from IBM, who is working on the Web Services Testing Forum (WSTF). ADL, IMS, and others have mechanisms for conformance testing that are primarily based on schema verification and schematron rules that go beyond XML. One item has been how do we test behaviors for web service implementations. Most learning technology organizations would rather use someone else's approach. WSTF seems like a good starting point.
Joel explained that MedBiquitous is a standards body in healthcare education, targeting continuing ed, competencies, making sure quality of care is improved. They work with medical schools, professional societies, regulatory bodies, etc. A main way of exchanging data is Web services. Web service guidelines have been developed; they have been used for other groups as well. MedBiq has an interest in interoperability and conformance testing. We do testing for XML, but haven't gotten to Web services testing. We are working up stack of WS*, not going further than we have to.
Doug introduced himself. He is from IBM and has worked with Joel in the past. He has worked on Web services for several years. He will provide an overview of wstf and what it can offer. He has thoughts on how we can leverage the forum.
Doug explained that existing standards committees, during testing, found few interoperability issues. It's because the tests are designed to verify "can they do what specs say they can do" rather than "Does it do what customer needs it to do?" The answer in reality is probably not.
Joel asked what was the reaction from OASIS, which had intended to do compliance testing. Doug replied he hasn't seen compliance stuff done in Oasis other than the testing within committees around the spec. Joel commented they may have fallen down on their original goal.
Doug added that WSI brings specs together and develops profiles, but they have had political problems. They can't work on anything until the spec has exited the standards body. That may not be early enough to fix a problem. As a result, its hard to see how one spec works with others out there. WSTF is where smaller organizations can do testing where big vendors will participate. Also, WSTF is a place where they can ask questions about how to use different specs together and get an unbiased answer.
Traditionally each company does its own interoperability testing. If they test with competitors in house, they have to set up those competing systems in house. They can only test with versions shipping now. If you find an issue, its too late; product has shipped. WSTF allows companies to identify and fix problems sooner rather than later.
The goal of wstf is to focus on development and testing of non trivial, customer driven scenarios. This can help with product configuration and testing. Participants can help identify problems, offer workarounds, and take issues to the appropriate standards body so it can be fixed in a standardized fashion.
WSTF is not a standards developer (SDO). They take issues to SDOs.
WSTF is developing a shared testbed. IBM can host shared endpoint of a web service, which means they don't have to give betas of websphere out to everyone. Oracle and others can test with it. This allows companies to discover bugs sooner rather than later. In addition it puts everyone on equal footing for testing and provides a community approach to dealing with problems.
The main deliverables for WSTF are scenarios, test cases, expectations, conformance tests, and any findings of testing. This provides guidance for customers. With each scenarios, client and server endpoints can be tested. Also, there is automated testing. Automated clients can be invoked nightly. You can see how people do as they change their code over time.
Each scenario has list of participants/endpoints. A participant can host ancillary info - configuration, users guides, etc. Can have code fixes there, too.
Joel asked about the website. Doug maintains the website himself. It's free to join.
Joel asked if he cares if they use ws or industry specs. Doug replied no. They are focused on soap based services. But there is no reason why someone could test non-soap services. They are starting to look at rest and cloud testing. Anything going over the net is up for grabs. You can test interoperability or understanding of your schema.
Most work is done in private, which allows people to discuss more freely. Voting is done in private, too. What happens in wstf stays within the group. Results can't be used by marketing. If you contribute something, you are providing a royalty free license. That prevents submarine patents. Scenarios can be made public if there are 5 implementations, 2/3 of which vote to go public. People who choose not to join can still test.
Dan asked when it goes public, are all members forced to go public? Who is behind the legal authority?
Doug explained participants are not listed. Individual companies may choose if their endpoints are listed. Each member is responsible for having a legal document signed, which goes to the WSTF site. There is no single governing authority. It is just an agreement between organizations.
They do announce scenarios, in some cases. IBM hosts the website. Participation is voluntary. The website, www.wstf.org, went live December 2008. The impact has been significant with regards to changes to specs and profiles. He added it would be great if we brought them scenarios. WSTF can serve as a sounding board.
Valerie asked Joel about applicability to MedBiquitous. Joel replied that we should look at the particulars. If there is something we want to replicate among a lot of groups, it would be a great venue for testing. If we are successful, that would give WSTF a way to show applicability to industry verticals.
Dan commented that from ADL perspective, they are interested in similar things. They are looking for neutral place for testing. He expressed concern that members might think to WSTF is too WS* centric. He needs to build consensus about using this approach. He will suggest WSTF at the mtg on the 12th. Doug replied that we are free to post slides where we like.
Joel commented he will be at an impact conference. Doug commented he won't be able to attend.
Doug agreed to send Valerie a paragraph to use in an announcement to let the MedBiquitous community know about WSTF.