The Australian Government coat of Arms

Communities of practice

Communities of practice

Some questions from Environment

Hi all,

The Dept of Environment are still getting their access sorted, but in anticipation of their arrival they would like to ask the following -

Some of the things we are interested in feedback on are below. Maybe someone can add them appropriately while my access is sorted out?

  1. What kinds of applications did other start with during the POC phase?
    a. For us I was thinking SAP could be one but with the shared services move the ROI and buy in may be affected. Have any others looked at automated regression for SAP and if so which tools did they use?
    b. Another was desktop patching/updates but I feel this could be a can of worms and require a tool which functions across different platforms. How do other manage O/S, Core Application, Security and MS Office updates?
    c. Online Services seems a logical first choice – browser based and MuleSoft between it and a CRM. Would you do a POC for this at the interface level or try and get developers support to leverage automating at the API layer?
    i. If anyone has done this are there tools / frameworks you think would be better than others and some we should simply avoid?
  2. With the development of automation tools there seems to be some arguments around purpose built framework versus out of the box plug and play type solutions. What types of frameworks do departments have in place?
  3. If a framework has been purpose build have others stuck with a Page Object Model or have some headed down the BDD path?
    a. What are the Pros and Cons of each that you have encountered?

Hi,

At Employment we had no official organisational POC, rather an enterprising project newly introduced to Agile experimented with a team POC that then got traction. The POC concerned was our ‘Employer’ system - a browser based UI back by a .Net API. Initial automation was via Selenium POMs with a very bare bones (early Alpha development stage) framework that had been setup by a Microsoft consultant but had then not seen much action to support it. The ‘Employer’ system had the advantage it was relatively small and isolated from the rest of the system while still being part of it (prior automation experiments were either conducted by a highly centralised support team and failed, or by a decentralised project team and succeeded but were so isolated from the main system they weren’t relevant to everyone else).

The success of this POC led to recruitment of a contractor to complete the bespoke framework, and continued experimentation with automation within the initial POC project team, this time expanding automation to the API layer which was even more successful. This API experiment eventually went beyond automation testing and into the realm of massive data generation.

Continued (mostly) successful experiments and the existence of the new framework led to some other teams getting aboard (and bringing in some other experiments that had also been occurring but been less well publicised) and eventually led to our current state of affairs with our internal Community of Practice and an attempt to grow automation organisation wide (whether we succeed is yet to be seen, tune in next year for more exciting episodes!).

So - should you target UI or API? UI is flashy and helps to bring people on board (seeing a Selenium ghost monkey type its way through screens is cool the first few times you see it) and traditionally what managers and testers think of when they think of automation. But UI automation is much clunkier than automating an API, is slower, and takes more work to maintain. API/Service automation also directly supports things like massive data and can also be leveraged to set up a status for a UI test to start from later. If your service layer is in good enough shape and you don’t need to sell automation, I say target that, or move to it as soon as possible after UI has sold the game.

Side note - the online patching guys are now very interested in our automation testing, though mainly to regression tests systems after patch is released.

I advocate a purpose built framework - but I have no experience with any other type. Bespoke does mean it can be customised to your exact requirements and you have no vendor lock-in, though up front cost is more.

Data Driven POMs. For us True Test Driven Development or Behaviour Driven Development is a future goal that goes beyond just automation. Even switching to automation is a big cultural challenge without also trying to switch to BDD at the same time. But your POMs should definitely be data driven with minimal hard-coding of testable values.

Note - if you set up a good data driven engine for your tests you can start to stick in randomisers for things like names, addresses, birthdates or whatever when you have the capacity - and now you get better variety in your test inputs, and less manual effort to create the data unless you want a targeted dataset of course (and you an mix and match even then, some random info, some targeted info).