What's up?

In MDL-65493 we found that there are a number of instances where behat steps are called directly from one another rather than using the correct $this->execute() syntax.

Show me

$ ag 'this\->i_' | ag '/behat/' | wc -l

39

Why does this matter?

When a step is called from a feature file via Mink, a number of extra things also happen. Specifically we respect a number of Hooks, and we perform certain error checking both before and after the step.

Part of that checking is to wait for JS to complete.

When we do not call steps using execute, we encounter random failures more often because those steps do not properly complete.

Can't this be done automatically?

Sadly not. We cannot inject these steps between function calls.

Well can't we at least detect these automatically?

We can't easily detect these yet, and there are situations where you can directly call class methods from one another. For example, you can use $this->find or $this->get_selected_node or $this->execute and countless others.

The only way we could check for these thoroughly is to look find all Behat definitions, then search for direct uses of those in the codebase, but even this would be incomplete in some situations.