VBS3 contains over 1,600 features, includes more than 12,000 models, and meets hundreds of different training and simulation use cases. The amount of VBS3 content and diverse ways VBS3 can be used poses a significant challenge for BISim’s testing and quality assurance team.
“We are building additional layers of testing framework, based on exact customer use cases,” says David Novotny, BISim’s Director of QA. “We gather example scenarios provided by customers along with an attached ‘screenplay’ describing how the training is executed.”
BISim has already received example scenarios from several enterprise-level customers to help implement the testing process and invites customers to submit sample scenarios to assist with testing. These greatly help the QA department focus on what’s really important to the customer.
The addition of customer scenarios during QA testing comes alongside a host of improvements in BISim product testing, which has been ongoing over the past few years. Not only has BISim’s QA team more than quadrupled since 2012, automated testing has increased significantly.
Since 2012, the QA team has grown from a team of seven to more than 30 staff today on three continents. And, the expectations at BISim for its QA team are higher than the industry at large. QA staff must be capable of performing at the junior developer level, able to debug code and suggest fixes to save developers time.
At BISim, automated testing has grown in scope and has become more sophisticated. In 2015, when automated testing began, there were around 100 feature scale tests completed that year. By 2017, the number of automated tests jumped to over 900, encompassing automated use case tests, stress tests, soak tests, performance tests, feature-scale component tests and more. BISim performs over 10,000 hours of automated testing per version and over 5,000 hours of manual testing per version. These efforts have led to continuous optimizations in performance and stability for each release of VBS3.
“To improve our processes, the BISim QA staff are in more discussions with customers, explaining our processes, and visiting customers on-site to gather data on how they train,” Novotny says.
In addition, BISim has embedded QA staff within development teams to have QA involved at the very beginning of the development process. “This improves the flow of work, improves QA’s awareness of customers and their needs, and shortens the communication loop,” Novotny says. “That way issues in newly developed technologies will be visible much earlier, allowing us to have more time to resolve them and provide a higher quality product to our customers.”
If you’re interested in providing example scenarios for BISim QA to include in our testing procedures, contact BISim’s QA team at usecases@bisimulations.com.
BISim is always on the lookout for new talent. Visit bisimulations.com/company/careers to see available positions in the QA department and other departments in the company.