Test Pyramids & The 3 Pillars of Observability

Richard Bradshaw- Test Pyramids

Richard originally came up with the ideas of "Test Pyramids" on his original tweet... https://twitter.com/friendlytester/status/1120118030292475904 @FriendlyTester

This is the Test Automation Pyramid

He discusses this more in his talk at Sauce Con 2019. 

 

https://www.youtube.com/watch?v=83aHKC5C14Q&list=PL67l1VPxOnT5PZQ1r60wQoT2UPDk1of4z

In his talk he discusses triangles, and pyramids.

The official blurb is...

In this SauceCon 2019 presentation, Richard discusses the importance of having a test automation strategy and provides a set of questions to help QA teams and developers understand the context required to formulate an effective automation strategy. He then reviews how to evaluate and challenge existing automation models to ensure test automation efforts are maximised.

Every Pyramid has a base, and theTestability is at the base. The whole strategy to automation is based around Testability. 

Team Context
Who is on your team? Languages? Developers? How do they contribute? Learn your team, Access to your team

Project Context

What framework should I use? What tools should I use? Get into the project first, know the tools. 

Product Context

What is the product? Knowledge of the product. Who is using the product

Technology

Frameworks/ Selenium/ React. What are you testing for the right tools. Apps and act accordingly. 

Stakeholders
Know your stakeholders, If automation is being pushed- ask why! What are they after to use appropriate tools. Are users using differently to automation? Need to back yourself up- justify your work. 

Infrastructure

Releases happen when? Realistic- how many environments do you have? Is there a pipeline in place/ can you add etc? Include that you need it adding to the pipeline. 

Now we need to decide what to automate. What do we care about?

These can be broken down into multiple different layers- does the team have the skills to automate the layers?

Targeted

Reliable

Informative

Maintainable

Speed

Credit: Mark Winteringham @2bittester

Checks at the top of the pyramid are harder/ cost more

If it doesn't look like a pyramid, it's OK. You just need to justify your choices. 

The Three Pillars of Observability

From Dynatrace (a full stack monitoring programme) & Lightstep

1. Metrics

Metrics are values that ex-press data about a system. These are counts/ measures and calculated over a period of time. 

• Examples such as...
• How much memory being used
• Requests per second

2. Traces

A single trace shows the activity for an individual transaction or request as is flows through an application. 

• Examples such as...
• Credit Card Validation
• Checking valid dates on a booking website

 3. Logs

Structured or unstructured lines of text that are emitted by an application in a response to an event in the code. Logs are "what happened"

Structured log example:

127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326

Unstructured log example:

"thing happened"