Selenium tests - a couple of loose thoughts...

Recently I've blogged about how to employ fluent interface when creating a framework for Selenium-based integration tests. After a few talks I gave I think I have a some grasp of what the major concerns for taking this approach are. Let's go quickly through what I observed was the top 5 questions.

Assertions shall be done in tests - not in page objects

This is probably the numero uno questions/concern voiced during the presentations. Let's assume for a second that we're not doing fluent interface in our Selenium tests and force ourselves to write assertions in tests, because the original documentation says so.
It turns out pretty quickly that there's more than one of the same assertion in more than one test. You may ask why that's the case? If you try to keep your scenarios separated and isolated you will end up either navigating to the page in question more than once (and verifying if the navigation succeeded). That's just one example but you get the idea.
For me personally the page object being the right place to define the assertion has more to do with scoping and avoiding unnecessary getter just to be able to assert a state on a given page. I know that for years we've been taught the POJO paradigm and the JavaBean convention, that most of us have that already in our DNA, so to speak. There are even things to spice up your life in this regard and have the compiler generate getters and setters for you. Hell, there are even languages like Groovy (my very favorite one) that will generate those things for you without you even asking for it. I actually like the idea because it takes completely away the POJO bloated boilerplate and gives you a sane way to express your intent. All that does not mean that we are allowed to turn off thinking and follow the convention like rats followed the pied piper. When you need an action on an object you don't create a separate friend/utility class for it but you embed that inside that object. I don't know if that's already domain-driven design or whatever but it is just plane sanity dictating it. The lack thereof can be easily observed in the Java Collection API (Collections.sort(people, comparator) instead of simply calling people.sort()). We've been fed this pulp for years - no wonder we take it for caviar.

I already have an existing suite of tests - what shall I do?

This is probably going to hit everyone who'd like to try the fluent interface in Selenium tests. Let me quote an old saying: "No pain, no gain". Things will not evolve by themselves - you need to take matters into your own hands and make the world a better place. I know that for many of you those tests are more of a pain in the ass that the boss/tech lead/company policy brutally makes you do but if you don't change your mindset how can you even begin to expect to start changing how others operate?
I say start small. Write one or two tests, create a handful of page objects, make a presentation, get others excited. The hard part is already done in all your other (for the sake of it I'm just going to call them legacy) tests and all that's left is just changing the outfit.

How do I allow writing custom assertions?

I brought this up as a separate concern because it's been made very clear for me that sticking with what the regular assertion framework gives you is far from optimal. Although AssertJ or just the assertions from org.junit.Assert class give a certain amount of flexibility it'd be nice to have some common patterns baked in into some custom assertion class.
Let's make one thing crystal clear: there are 2 reasons why you'd want to do it. One, when you want to have for example strongly-typed assertions that compare domain objects, maybe you parse and assert strings in a peculiar way throughout your test suite - that's the generic extension and it is perfectly legit. However there is a whole group of other reasons why people do it: to write custom assertions that access the state of your page objects and assert if a given action gave proper results. Well that's precisely the Collections.sort(people, comparator) vs people.sort(comparator) thing and if you do it that old dumb way you're just going to have to stick with it while the whole galaxy moves forward.
That being said there's absolutely nothing (besides sanity) preventing you from writing a separate custom assertion and calling it from a wrapper inside a page object to continue the fluent interface.

Does it work for you?

It's been almost a year when this pattern has been used in our product. During that time we went from near-no-tests to around 60% coverage of a very legacy system which in my opinion counts as a great success. But the best part of it is that it doesn't stop! Developers simply love the way they create tests. It's easy and now that we have a lot of the page objects already created writing new tests is really simple. Even simple bug fixes begin with a selenium test and it takes mare minutes to write them.
And above all I love the way it translates from steps to reproduce to steps in a test - it is plain awesome! Please bear in mind we're talking about things in one of the most constrained languages out there that a moderately sober student can learn in a week!

Are there any drawbacks?

Yes, there are! Since creating data for tests is so extremely easy developers tend to prepare that data every time they write a test and that tends to make those tests run slowly. So if speed is what you're craving for you need to be smart about it. We tend to create separate database migrations that run before selenium tests and that way if we already have part of the functionality covered in other tests we can speed things up.

How long runs your test suite?

Around 5 hours. That being said doing the same thing manually takes a week so I say we're well on the safe side. Please bear in mind that we're executing those tests sequentially and that doesn't need to stay this way. Since most of our tests have data specific for them created before each test that makes it very much possible to run them in parallel. Initial tests show that running the whole system on developer's notebook and executing 20 tests in parallel cuts the time to 20 minutes which is well within the realm of modern continuous deployment scenarios.

Well, that's it! Happy coding!