In this QualiFYI episode, Christine Bentsen, Head of Product Marketing of Continuous Testing at Broadcom joins me to discuss the rise of continuous testing, the impact it has on DevOps, and how to alleviate bottlenecks using automated test environments to continuously improve quality. Check out the full video and transcript below!
How to Alleviate Continuous Testing Bottlenecks with Automated Testing Environments
Q: Let's get right into it. You are a thought leader extraordinaire of continuous testing. Tell us what is continuous testing?
A: Continuous testing is the newest way to do testing. Traditionally, testing was done as a time-boxed event at the end of the software delivery lifecycle. It was done by a team of dedicated testers, and it happened at the very end. Right?
A time-boxed event.
Now, with the advent of new practices like DevOps and continuous delivery in Agile, that causes a bottleneck because it happens at the end.
What continuous testing does is it shifts testing and the idea of quality left into planning and Dev and right into production and using data from the application lifecycle from the pipeline to continuously improve quality and to make everyone responsible for quality. This also allows you to find and fix problems as soon as they happen in the application delivery life cycle when they're easier to identify and faster and cheaper to fix.
Q: You mentioned bottlenecks. You mentioned finding the problems right at the right time, but surely there are some common challenges that people face in continuous testing. Can you walk us through some?
A: Yes, absolutely. So of course companies that are trying to do DevOps and continuous delivery, and that are having these bottlenecks, and they see that the bottleneck face testing, of course they're like, well let's do continuous testing. That makes a lot of sense.
But they have trouble adopting continuous testing, and one of the common problems is that the tools that those traditional testing teams have been using, they're typically built with a UI, they're a little bit clunky.
They were built for different teams, right? Sometimes the licensing is restrictive, what they support is restrictive, and they just don't work for the way that these Agile teams and Dev testers want to work.
What these folks want to do is they want to do things as code. They want to do it in their IDEs without having to switch to another application. They want it embedded in the CI/CD tool chain, right. And they want to work with open source.
All of these things do not work with those legacy testing tools. Then we also have some problems with continuous testing bottlenecks itself.
Right. Every year, Broadcom does a report with Sogeti, the continuous testing report, and we're about to release the new one, and I'll give you a couple of previews about what's going to be in this one.
So year over year we found that companies are still struggling with continuous testing. They're still struggling in a lot of the same areas. My summary of the report was everything is hard and we can't figure it out. Right? But there are a lot of problems and we asked them, what are the chief problems that you have?
Things that they told us is they have trouble getting visibility into the pipeline.
They have trouble assessing performance from the end user standpoint, like when you release the application under peak load or from global perspectives, what's that going to look like to your end user? What's the impact on your end user? They have trouble with test environments, right? They spend a lot of time trying to get those test environments right. There's a lot of things that happen actually within... Oh I forgot one, one of the things that they all told us, many people tell us in the report that they're doing continuous testing, but then when you drill down into it, they say they're having trouble achieving testing in Sprint, which is continuous testing.
There's a lot of disconnects, and like I said, a lot of companies are really struggling with making continuous testing work. But back to the test environment piece, right?
So this is a stat that we see, not only in this report, but I see it in lots of different reports, how much time and effort companies are spending trying to get test environments on demand, spin them up and spin them down when they need them.
Q: I'm so happy you bring that up because that's exactly what we're into. How important then is automated environment provisioning?
A: Drilling down into that report a little bit more, and again this is a sneak preview into what is coming soon.
Yes. The respondents told us that they're spending 42% of their time, their team's time building, and maintaining, and decommissioning these test environments.
If you think about how much time is being spent on this, if you took that time and you turned that effort into innovation or doing other things that were adding value to your bottom line.
What would that do in your organization? How would that change your organization? It would be fantastic, right?
And so another thing we asked them is what would help you achieve more continuous testing maturity? 40% of them said automated test environments.
Q: It sounds like there's a lot that's going to be happening in the future of continuous testing then if you're uncovering all of these issues and bottlenecks, can you give us a glimpse at what we can look forward to?
A: I'm sure that you have heard that AI is a big thing. Everybody is talking about AI. And of course we're going to have to test AI systems.
That's really big. But not only are we going to have to test AI systems, but the technology of AI is really applicable and going to be fantastic for continuous testing. Instead of running all of the tests all the time to ensure quality, you can now have AI help you identify which tests you need to run in which order. It can help synthesize data from across multiple applications that you have in your organization and say, ah, here are some problems that you could address to raise quality. There's just a lot of different things that AI can help you with to make your test cycles more efficient and more accurate, and so you use resources more wisely, and ultimately you improve quality.
But before we do that, just stepping back, the next big thing, I think, to make continuous testing more of a reality and to help more companies achieve that maturity is we need to make continuous testing easier to adopt.
Okay. How do you do that?
Yeah, so you have all these companies that say, well, continuous testing looks good, but we're not able to do it. Making continuous testing easy, having an integrated platform, having it work in the UI, and also for shift left teams.
You have one application where you can share reports, you can share services, because you're going to shift left, but probably not all at one time.
So some of your teams are still going to be doing traditional testing, being able to support open source, having all of the applications or all the capabilities that you need at your fingertips, and I know a place that you can get an application like that.
That's what we really want to do is make continuous testing easy for organizations to adopt.
And with quality, because you mentioned that word quite a few times.
Exactly. It all comes back to your business outcomes.
We're not doing continuous testing just to do more testing faster.
We're doing it to help you achieve your business outcomes, and those business outcomes are being able to respond to marketplace opportunities, being really nimble, and being able to do that with confidence, and the way to do that is through continuous testing.
There's a lot coming out, and being able to achieve that quality and speed, it's going to change the way we do business.
Well, thank you so much for your time today, Christine. It was so much fun talking to you as always, and learning more about continuous testing.
Visit Quali.com to learn more about Environment as a Service.
